Jul 10 19:05:21.307197 ip-10-0-135-73 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 19:05:21.307210 ip-10-0-135-73 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 19:05:21.307219 ip-10-0-135-73 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 19:05:21.307567 ip-10-0-135-73 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 19:05:31.314779 ip-10-0-135-73 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 19:05:31.314794 ip-10-0-135-73 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 18395ad3157d4c7796af92698105c086 -- Jul 10 19:07:27.853220 ip-10-0-135-73 systemd[1]: Starting Kubernetes Kubelet... Jul 10 19:07:28.270774 ip-10-0-135-73 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 19:07:28.270774 ip-10-0-135-73 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 19:07:28.270774 ip-10-0-135-73 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 19:07:28.270774 ip-10-0-135-73 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 19:07:28.270774 ip-10-0-135-73 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 19:07:28.271995 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.271547 2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 19:07:28.277376 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277353 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:07:28.277376 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277375 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:07:28.277376 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277379 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277382 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277386 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277388 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277391 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277393 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277396 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277400 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277403 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277405 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277408 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277410 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277413 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277415 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277418 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277421 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277423 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277428 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277432 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:07:28.277487 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277435 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277439 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277442 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277444 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277447 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277449 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277452 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277454 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277457 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277459 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277461 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277465 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277467 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277469 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277472 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277476 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277479 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277481 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277484 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:07:28.277956 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277488 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277492 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277496 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277500 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277503 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277506 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277508 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277511 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277514 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277516 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277519 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277522 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277525 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277527 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277530 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277532 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277535 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277537 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277540 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277542 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:07:28.278423 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277545 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277549 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277551 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277554 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277556 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277558 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277561 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277563 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277567 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277569 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277572 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277575 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277577 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277580 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277582 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277585 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277588 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277590 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277595 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277599 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:07:28.279019 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277601 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277606 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277608 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277611 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277613 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277616 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277619 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277622 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277624 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277627 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277629 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277633 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277636 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277639 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277642 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277644 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277647 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277649 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.277652 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279163 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:07:28.279501 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279172 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279176 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279179 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279182 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279185 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279188 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279192 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279194 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279199 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279203 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279206 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279209 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279212 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279215 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279217 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279220 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279223 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279225 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279228 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:07:28.280011 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279230 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279233 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279235 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279238 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279242 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279244 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279247 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279250 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279252 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279255 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279257 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279260 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279263 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279265 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279268 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279270 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279273 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279276 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279278 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279281 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:07:28.280526 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279283 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279286 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279289 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279291 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279294 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279296 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279299 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279301 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279303 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279306 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279308 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279312 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279314 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279317 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279319 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279321 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279324 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279328 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279331 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:07:28.281045 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279334 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279337 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279339 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279342 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279344 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279347 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279349 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279351 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279354 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279356 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279359 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279361 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279364 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279366 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279369 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279372 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279377 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279380 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279383 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279385 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:07:28.281509 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279388 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279390 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279393 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279395 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279400 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279404 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279407 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279409 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279412 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279414 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279417 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279419 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279421 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279424 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279426 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279429 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279432 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279434 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279437 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279440 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:07:28.282015 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279833 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279838 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279841 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279843 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279846 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279848 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279851 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279853 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279856 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279861 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279864 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279883 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279886 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279889 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279891 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279894 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279897 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279900 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279903 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:07:28.282498 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279905 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279908 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279910 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279913 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279915 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279918 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279920 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279923 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279925 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279928 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279930 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279933 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279936 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279939 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279942 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279944 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279947 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279950 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279952 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:07:28.282964 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279954 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279957 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279961 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279965 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279967 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279970 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279973 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279975 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279978 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279980 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279983 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279985 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279988 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279990 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279993 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.279995 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280001 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280004 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280006 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280009 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:07:28.283421 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280011 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280014 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280017 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280020 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280023 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280025 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280028 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280030 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280033 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280035 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280038 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280040 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280043 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280045 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280048 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280050 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280053 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280056 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280059 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280062 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:07:28.283909 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280064 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280067 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280071 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280073 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280077 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280080 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280083 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280086 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280092 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280095 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280098 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280100 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280103 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280105 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280108 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280110 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280113 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280115 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280118 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280120 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:07:28.284387 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280123 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280203 2556 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280212 2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280222 2556 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280229 2556 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280235 2556 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280239 2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280246 2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280252 2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280257 2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280260 2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280263 2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280267 2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280270 2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280275 2556 flags.go:64] FLAG: --cgroup-root="" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280278 2556 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280281 2556 flags.go:64] FLAG: --client-ca-file="" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280284 2556 flags.go:64] FLAG: --cloud-provider="external" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280287 2556 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280292 2556 flags.go:64] FLAG: --cluster-domain="" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280297 2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280300 2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280303 2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280306 2556 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 19:07:28.284865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280310 2556 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280313 2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280317 2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280320 2556 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280323 2556 flags.go:64] FLAG: --contention-profiling="false" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280326 2556 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280329 2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280332 2556 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280335 2556 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280340 2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280342 2556 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280345 2556 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280348 2556 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280351 2556 flags.go:64] FLAG: --enable-server="true" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280354 2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280358 2556 flags.go:64] FLAG: --event-burst="100" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280361 2556 flags.go:64] FLAG: --event-qps="50" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280364 2556 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280368 2556 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280371 2556 flags.go:64] FLAG: --eviction-hard="" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280375 2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280377 2556 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280381 2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280384 2556 flags.go:64] FLAG: --eviction-soft="" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280387 2556 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 19:07:28.285479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280390 2556 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280393 2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280395 2556 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280400 2556 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280403 2556 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280406 2556 flags.go:64] FLAG: --feature-gates="" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280410 2556 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280413 2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280416 2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280419 2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280422 2556 flags.go:64] FLAG: --healthz-port="10248" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280425 2556 flags.go:64] FLAG: --help="false" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280428 2556 flags.go:64] FLAG: --hostname-override="ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280431 2556 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280434 2556 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280437 2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280441 2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280444 2556 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280446 2556 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280449 2556 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280452 2556 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280455 2556 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280458 2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280462 2556 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 19:07:28.286106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280465 2556 flags.go:64] FLAG: --kube-reserved="" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280467 2556 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280471 2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280474 2556 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280477 2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280481 2556 flags.go:64] FLAG: --lock-file="" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280484 2556 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280487 2556 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280490 2556 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280496 2556 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280498 2556 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280503 2556 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280505 2556 flags.go:64] FLAG: --logging-format="text" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280508 2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280511 2556 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280514 2556 flags.go:64] FLAG: --manifest-url="" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280517 2556 flags.go:64] FLAG: --manifest-url-header="" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280522 2556 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280525 2556 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280529 2556 flags.go:64] FLAG: --max-pods="110" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280532 2556 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280535 2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280538 2556 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280540 2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280544 2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 19:07:28.286690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280546 2556 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280549 2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280557 2556 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280560 2556 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280563 2556 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280566 2556 flags.go:64] FLAG: --pod-cidr="" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280569 2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280575 2556 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280578 2556 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280581 2556 flags.go:64] FLAG: --pods-per-core="0" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280584 2556 flags.go:64] FLAG: --port="10250" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280587 2556 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280590 2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06f53e3a1d4cc3986" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280594 2556 flags.go:64] FLAG: --qos-reserved="" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280597 2556 flags.go:64] FLAG: --read-only-port="10255" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280600 2556 flags.go:64] FLAG: --register-node="true" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280602 2556 flags.go:64] FLAG: --register-with-taints="" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280606 2556 flags.go:64] FLAG: --registry-burst="10" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280609 2556 flags.go:64] FLAG: --registry-qps="5" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280612 2556 flags.go:64] FLAG: --reserved-cpus="" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280614 2556 flags.go:64] FLAG: --reserved-memory="" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280618 2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280621 2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280624 2556 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280627 2556 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 19:07:28.287308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280630 2556 flags.go:64] FLAG: --runonce="false" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280633 2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280636 2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280639 2556 flags.go:64] FLAG: --seccomp-default="false" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280642 2556 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280644 2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280647 2556 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280650 2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280653 2556 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280656 2556 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280659 2556 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280661 2556 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280664 2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280667 2556 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280670 2556 flags.go:64] FLAG: --system-cgroups="" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280672 2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280678 2556 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280681 2556 flags.go:64] FLAG: --tls-cert-file="" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280684 2556 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280688 2556 flags.go:64] FLAG: --tls-min-version="" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280691 2556 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280694 2556 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280696 2556 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280699 2556 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280702 2556 flags.go:64] FLAG: --v="2" Jul 10 19:07:28.287935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280706 2556 flags.go:64] FLAG: --version="false" Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280710 2556 flags.go:64] FLAG: --vmodule="" Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280714 2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.280718 2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280842 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280845 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280848 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280852 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280855 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280858 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280860 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280862 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280865 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280887 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280890 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280892 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280895 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280898 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280900 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280903 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280905 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:07:28.288537 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280908 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280910 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280913 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280915 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280918 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280921 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280924 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280927 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280929 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280932 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280934 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280937 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280940 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280942 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280945 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280947 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280950 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280952 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280955 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:07:28.289085 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280959 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280963 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280966 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280969 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280971 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280974 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280976 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280979 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280981 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280984 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280986 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280988 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280991 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280993 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280996 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.280998 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281000 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281003 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281006 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281009 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:07:28.289558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281011 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281013 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281016 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281018 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281021 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281023 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281025 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281028 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281030 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281032 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281035 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281038 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281040 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281043 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281045 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281048 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281052 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281055 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281058 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:07:28.290126 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281062 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281065 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281067 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281070 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281074 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281077 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281082 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281084 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281088 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281091 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281094 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281097 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281100 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281103 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281106 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281108 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281111 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281113 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281116 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281118 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:07:28.290587 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281121 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281124 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281126 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.281129 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.281837 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.289275 2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.289294 2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.289316 2556 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.289327 2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289401 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289407 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289411 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289416 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289419 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:07:28.291089 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289422 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289425 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289428 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289431 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289433 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289436 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289439 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289441 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289444 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289446 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289449 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289452 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289454 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289457 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289459 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289462 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289465 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289467 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289470 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289473 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:07:28.291441 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289476 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289478 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289481 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289483 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289487 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289489 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289492 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289494 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289496 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289499 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289501 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289504 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289506 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289509 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289511 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289513 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289516 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289518 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289521 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:07:28.291945 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289523 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289525 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289528 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289531 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289534 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289538 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289541 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289544 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289546 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289548 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289551 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289554 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289556 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289559 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289562 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289564 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289567 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289570 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289573 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:07:28.292397 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289575 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289578 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289580 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289582 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289585 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289587 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289590 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289592 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289594 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289597 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289599 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289601 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289604 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289607 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289609 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289611 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289614 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289617 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289619 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289622 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:07:28.292858 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289624 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289627 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289629 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289632 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289634 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289637 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289639 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289642 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289646 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289649 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289652 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289655 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289658 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289661 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289663 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:07:28.293350 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289666 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.289671 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289790 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289796 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289799 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289801 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289804 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289806 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289809 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289811 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289814 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289816 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289819 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289822 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289824 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 19:07:28.293718 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289827 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289829 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289832 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289834 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289837 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289841 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289845 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289848 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289851 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289854 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289856 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289859 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289861 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289864 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289884 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289894 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289898 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289902 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289905 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289908 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 19:07:28.294285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289913 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289915 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289918 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289921 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289924 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289926 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289928 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289932 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289935 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289938 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289941 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289943 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289946 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289948 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289951 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289953 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289956 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289958 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289962 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 19:07:28.294764 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289967 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289970 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289973 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289976 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289978 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289982 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289984 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289987 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289991 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289994 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289996 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.289999 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290001 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290003 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290006 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290008 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290010 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290013 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290015 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290018 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 19:07:28.295245 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290020 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290022 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290025 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290027 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290029 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290032 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290034 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290036 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290039 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290041 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290044 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290046 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290049 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290051 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290054 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290056 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290059 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290061 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290063 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290066 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 19:07:28.295747 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290068 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290070 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290073 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290076 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290078 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290081 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:28.290083 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.290089 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.290232 2556 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.293496 2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.294686 2556 server.go:1019] "Starting client certificate rotation" Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.294798 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 19:07:28.296297 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.294845 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 19:07:28.297223 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.297202 2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 19:07:28.304593 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.304560 2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 19:07:28.315747 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.315722 2556 log.go:25] "Validated CRI v1 runtime API" Jul 10 19:07:28.318028 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.318010 2556 log.go:25] "Validated CRI v1 image API" Jul 10 19:07:28.318861 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.318842 2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 19:07:28.323663 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.323634 2556 fs.go:135] Filesystem UUIDs: map[2c8125f6-8fc2-4fbd-8757-e44ee8e3c623:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d289fed0-cf95-4568-b8f6-6cd928457ff0:/dev/nvme0n1p3] Jul 10 19:07:28.323770 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.323664 2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 19:07:28.329339 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.329216 2556 manager.go:216] Machine: {Timestamp:2026-07-10 19:07:28.32781184 +0000 UTC m=+0.344270208 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099067 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec211632725888c0eabcabae92624cdd SystemUUID:ec211632-7258-88c0-eabc-abae92624cdd BootID:18395ad3-157d-4c77-96af-92698105c086 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:55:cb:a1:45:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:55:cb:a1:45:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:12:a6:43:9e:b9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 19:07:28.329339 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.329325 2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 19:07:28.329509 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.329413 2556 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 19:07:28.329724 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.329708 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 19:07:28.331913 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.331864 2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 19:07:28.332065 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.331918 2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-73.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 19:07:28.332644 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.332632 2556 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 19:07:28.332677 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.332647 2556 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 19:07:28.332677 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.332654 2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 19:07:28.333373 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.333361 2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 19:07:28.335498 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.335485 2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 19:07:28.338955 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.338939 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 10 19:07:28.339089 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.339079 2556 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 19:07:28.339135 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.339112 2556 kubelet.go:486] "Attempting to sync node with API server" Jul 10 19:07:28.339135 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.339131 2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 19:07:28.339204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.339143 2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 19:07:28.339204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.339152 2556 kubelet.go:398] "Adding apiserver pod source" Jul 10 19:07:28.339204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.339162 2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 19:07:28.339582 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.339565 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 19:07:28.339582 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.339584 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 19:07:28.341731 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.341708 2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 19:07:28.343660 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.343645 2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 19:07:28.343727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.343677 2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 19:07:28.343906 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.343894 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 19:07:28.343943 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.343911 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 19:07:28.343943 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.343917 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 19:07:28.343943 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.343923 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 19:07:28.343943 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.343931 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 19:07:28.343943 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.343936 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 19:07:28.344446 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.344435 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 19:07:28.344446 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.344446 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 19:07:28.344503 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.344460 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 19:07:28.344503 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.344466 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 19:07:28.344503 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.344475 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 19:07:28.344503 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.344483 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 19:07:28.344601 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.344544 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 19:07:28.344601 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.344550 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 19:07:28.348188 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.348170 2556 server.go:1268] "Started kubelet" Jul 10 19:07:28.349225 ip-10-0-135-73 systemd[1]: Started Kubernetes Kubelet. Jul 10 19:07:28.352258 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.348372 2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 19:07:28.352539 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.352439 2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 19:07:28.352690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.352668 2556 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 19:07:28.353827 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.353805 2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 19:07:28.354136 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.354062 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 19:07:28.354242 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.354130 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-73.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 19:07:28.354242 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.353842 2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-73.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 19:07:28.356682 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.356639 2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 19:07:28.356777 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.356694 2556 server.go:310] "Adding debug handlers to kubelet server" Jul 10 19:07:28.358432 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.357511 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-73.ec2.internal.18c10333c7f16df9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-73.ec2.internal,UID:ip-10-0-135-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-73.ec2.internal,},FirstTimestamp:2026-07-10 19:07:28.348130809 +0000 UTC m=+0.364589170,LastTimestamp:2026-07-10 19:07:28.348130809 +0000 UTC m=+0.364589170,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-73.ec2.internal,}" Jul 10 19:07:28.362319 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.362299 2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 19:07:28.362424 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.362318 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 19:07:28.362424 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.362416 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:28.363048 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363028 2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 19:07:28.363160 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363043 2556 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 19:07:28.363160 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363163 2556 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 19:07:28.363315 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363256 2556 factory.go:55] Registering systemd factory Jul 10 19:07:28.363315 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363270 2556 factory.go:223] Registration of the systemd container factory successfully Jul 10 19:07:28.363553 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363382 2556 reconstruct.go:97] "Volume reconstruction finished" Jul 10 19:07:28.363553 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363452 2556 reconciler.go:29] "Reconciler: start to sync state" Jul 10 19:07:28.363907 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363892 2556 factory.go:153] Registering CRI-O factory Jul 10 19:07:28.363998 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363922 2556 factory.go:223] Registration of the crio container factory successfully Jul 10 19:07:28.364050 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.363999 2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 19:07:28.364050 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.364024 2556 factory.go:103] Registering Raw factory Jul 10 19:07:28.364050 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.364039 2556 manager.go:1196] Started watching for new ooms in manager Jul 10 19:07:28.364404 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.364390 2556 manager.go:349] Starting recovery of all containers Jul 10 19:07:28.373351 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.373135 2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-73.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 19:07:28.373351 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.373173 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 19:07:28.375485 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.375463 2556 manager.go:354] Recovery completed Jul 10 19:07:28.380320 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.380298 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pndvq" Jul 10 19:07:28.380433 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.380421 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:07:28.383932 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.383915 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:07:28.384004 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.383948 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:07:28.384004 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.383964 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:07:28.384578 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.384558 2556 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 19:07:28.384578 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.384571 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:07:28.384578 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.384578 2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 19:07:28.384763 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.384598 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 10 19:07:28.385456 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.385440 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:07:28.385516 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.385468 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:07:28.385516 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.385497 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:07:28.386200 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.386186 2556 policy_none.go:49] "None policy: Start" Jul 10 19:07:28.386246 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.386209 2556 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 19:07:28.386246 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.386222 2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 19:07:28.387421 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.387386 2556 policy_none.go:47] "Start" Jul 10 19:07:28.388650 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.388633 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pndvq" Jul 10 19:07:28.394715 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.394583 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-73.ec2.internal.18c10333ca13b5f6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-73.ec2.internal,UID:ip-10-0-135-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-73.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-73.ec2.internal,},FirstTimestamp:2026-07-10 19:07:28.383931894 +0000 UTC m=+0.400390256,LastTimestamp:2026-07-10 19:07:28.383931894 +0000 UTC m=+0.400390256,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-73.ec2.internal,}" Jul 10 19:07:28.424708 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.424689 2556 manager.go:337] "Starting Device Plugin manager" Jul 10 19:07:28.424821 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.424736 2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 19:07:28.424821 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.424748 2556 server.go:87] "Starting device plugin registration server" Jul 10 19:07:28.425081 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.425067 2556 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 19:07:28.425159 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.425082 2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 19:07:28.425241 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.425219 2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 19:07:28.425373 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.425342 2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 19:07:28.425373 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.425358 2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 19:07:28.425958 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.425914 2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 19:07:28.426035 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.425979 2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:28.508928 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.508860 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 19:07:28.510645 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.510621 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 19:07:28.510758 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.510671 2556 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 19:07:28.510758 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.510697 2556 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 19:07:28.510758 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.510740 2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 19:07:28.513247 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.513219 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:07:28.525699 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.525619 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:07:28.527019 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.527000 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:07:28.527104 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.527036 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:07:28.527104 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.527047 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:07:28.527104 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.527077 2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.536885 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.536840 2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.536950 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.536888 2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-73.ec2.internal\": node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:28.570805 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.570768 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:28.611170 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.611128 2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal"] Jul 10 19:07:28.611242 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.611225 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:07:28.612386 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.612368 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:07:28.612458 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.612401 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:07:28.612458 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.612413 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:07:28.614008 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.613990 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:07:28.614125 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.614107 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.614162 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.614146 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:07:28.614818 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.614800 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:07:28.614910 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.614839 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:07:28.614910 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.614850 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:07:28.614910 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.614800 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:07:28.615007 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.614932 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:07:28.615007 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.614944 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:07:28.616377 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.616362 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.616428 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.616393 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 19:07:28.617229 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.617211 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientMemory" Jul 10 19:07:28.617320 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.617249 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 19:07:28.617320 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.617264 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeHasSufficientPID" Jul 10 19:07:28.641072 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.641040 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-73.ec2.internal\" not found" node="ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.646098 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.646076 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-73.ec2.internal\" not found" node="ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.665806 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.665769 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e39119c249ba3570b4866afa144c7646-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal\" (UID: \"e39119c249ba3570b4866afa144c7646\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.665806 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.665805 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e39119c249ba3570b4866afa144c7646-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal\" (UID: \"e39119c249ba3570b4866afa144c7646\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.666040 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.665823 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24c9a12d95ac9c34c371cd5589b6402d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-73.ec2.internal\" (UID: \"24c9a12d95ac9c34c371cd5589b6402d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.670970 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.670941 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:28.766723 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.766687 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e39119c249ba3570b4866afa144c7646-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal\" (UID: \"e39119c249ba3570b4866afa144c7646\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.766813 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.766724 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e39119c249ba3570b4866afa144c7646-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal\" (UID: \"e39119c249ba3570b4866afa144c7646\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.766813 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.766743 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24c9a12d95ac9c34c371cd5589b6402d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-73.ec2.internal\" (UID: \"24c9a12d95ac9c34c371cd5589b6402d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.766813 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.766794 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e39119c249ba3570b4866afa144c7646-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal\" (UID: \"e39119c249ba3570b4866afa144c7646\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.766984 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.766807 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e39119c249ba3570b4866afa144c7646-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal\" (UID: \"e39119c249ba3570b4866afa144c7646\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.766984 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.766860 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24c9a12d95ac9c34c371cd5589b6402d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-73.ec2.internal\" (UID: \"24c9a12d95ac9c34c371cd5589b6402d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.771095 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.771060 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:28.871822 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.871738 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:28.948840 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.948797 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.949985 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:28.949965 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal" Jul 10 19:07:28.972107 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:28.972061 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:29.072571 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:29.072525 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:29.173034 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:29.172955 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:29.215849 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.215817 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:07:29.274024 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:29.273990 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:29.294374 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.294341 2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 19:07:29.294536 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.294511 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 19:07:29.294577 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.294553 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 19:07:29.374974 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:29.374939 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-73.ec2.internal\" not found" Jul 10 19:07:29.392625 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.392563 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 19:02:29 +0000 UTC" deadline="2028-04-15 10:49:16.307744688 +0000 UTC" Jul 10 19:07:29.392625 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.392621 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15471h41m46.915128169s" Jul 10 19:07:29.445528 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.445436 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:07:29.463716 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.463679 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" Jul 10 19:07:29.478411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.478382 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 19:07:29.479549 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.479533 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal" Jul 10 19:07:29.488935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.488906 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 19:07:29.556789 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.556768 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:07:29.658817 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:29.658763 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39119c249ba3570b4866afa144c7646.slice/crio-d9dff77b280bce218dbd035d8cf5fb607d41ab43ba9abee84e72c4a4b18b169b WatchSource:0}: Error finding container d9dff77b280bce218dbd035d8cf5fb607d41ab43ba9abee84e72c4a4b18b169b: Status 404 returned error can't find the container with id d9dff77b280bce218dbd035d8cf5fb607d41ab43ba9abee84e72c4a4b18b169b Jul 10 19:07:29.659253 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:29.659226 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c9a12d95ac9c34c371cd5589b6402d.slice/crio-a2add955a3aad6f4024b0e616b5a948448366a6f351c924241a05e19ddfe6a4a WatchSource:0}: Error finding container a2add955a3aad6f4024b0e616b5a948448366a6f351c924241a05e19ddfe6a4a: Status 404 returned error can't find the container with id a2add955a3aad6f4024b0e616b5a948448366a6f351c924241a05e19ddfe6a4a Jul 10 19:07:29.668073 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:29.668053 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:07:30.340442 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.340386 2556 apiserver.go:52] "Watching apiserver" Jul 10 19:07:30.356534 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.356498 2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 19:07:30.358811 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.358772 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-7ng4r","openshift-network-operator/iptables-alerter-6tccc","kube-system/global-pull-secret-syncer-h42d9","kube-system/konnectivity-agent-fqzkj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk","openshift-cluster-node-tuning-operator/tuned-dk4vx","openshift-multus/multus-additional-cni-plugins-k4fbp","openshift-multus/multus-rcvkj","openshift-network-diagnostics/network-check-target-cn5r4","openshift-ovn-kubernetes/ovnkube-node-29hgv","kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal","openshift-dns/node-resolver-ctm9t","openshift-image-registry/node-ca-p67tg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal"] Jul 10 19:07:30.361058 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.361028 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.364862 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.363522 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.364862 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.364056 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5b5qp\"" Jul 10 19:07:30.364862 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.364210 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 19:07:30.364862 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.364352 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 19:07:30.364862 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.364524 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 19:07:30.364862 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.364645 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 19:07:30.365302 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.364886 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 19:07:30.365302 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.364937 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 19:07:30.366906 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.365711 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 19:07:30.366906 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.366280 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 19:07:30.366906 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.366709 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 19:07:30.366906 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.366728 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v8qcr\"" Jul 10 19:07:30.368635 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.368615 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.368758 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.368697 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:30.370201 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.370175 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:30.372865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.372709 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 19:07:30.372865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.372731 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 19:07:30.372865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.372833 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-76spq\"" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374395 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374593 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-cnibin\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374625 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-cni-binary-copy\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374654 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374682 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374711 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm4gj\" (UniqueName: \"kubernetes.io/projected/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-kube-api-access-sm4gj\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374743 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-os-release\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374792 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46f8b6c0-73ab-4684-91c6-04e358626759-host\") pod \"node-ca-p67tg\" (UID: \"46f8b6c0-73ab-4684-91c6-04e358626759\") " pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374821 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4qgt\" (UniqueName: \"kubernetes.io/projected/46f8b6c0-73ab-4684-91c6-04e358626759-kube-api-access-d4qgt\") pod \"node-ca-p67tg\" (UID: \"46f8b6c0-73ab-4684-91c6-04e358626759\") " pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.374916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.374853 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-kubelet-config\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.375501 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.375017 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.375644 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.375618 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-system-cni-dir\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.375740 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.375661 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46f8b6c0-73ab-4684-91c6-04e358626759-serviceca\") pod \"node-ca-p67tg\" (UID: \"46f8b6c0-73ab-4684-91c6-04e358626759\") " pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.375740 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.375687 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-dbus\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.375740 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.375713 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.375914 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.375741 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.375914 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.375863 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.377235 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.377213 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:30.377348 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.377320 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:30.377694 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.377673 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-99pk4\"" Jul 10 19:07:30.377787 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.377728 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 19:07:30.377858 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.377676 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 19:07:30.378027 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.378012 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 19:07:30.378432 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.378405 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jlzcw\"" Jul 10 19:07:30.378638 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.378623 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:07:30.378706 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.378642 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 19:07:30.378864 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.378850 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:30.378961 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.378938 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:30.380439 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.380418 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.382842 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.382510 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.383988 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.383916 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9rcm4\"" Jul 10 19:07:30.384293 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.384275 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 19:07:30.384566 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.384549 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.391807 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.386526 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.392328 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.387193 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 19:07:30.392328 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.389546 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:07:30.392328 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.390144 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 19:07:30.392328 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.390229 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 19:07:30.392328 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.390341 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 19:07:30.392772 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.390413 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zhn8c\"" Jul 10 19:07:30.392772 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.390524 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 19:07:30.392772 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.390649 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 19:07:30.392772 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.390803 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 19:07:30.395213 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.395188 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 19:07:30.395426 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.395410 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gzr28\"" Jul 10 19:07:30.395636 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.395620 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 19:07:30.396646 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.395910 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-j546m\"" Jul 10 19:07:30.396646 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.396565 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 19:07:30.401293 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.401267 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qzs2l" Jul 10 19:07:30.410120 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.409704 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qzs2l" Jul 10 19:07:30.464527 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.464500 2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 19:07:30.476216 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476180 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-lib-modules\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.476400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476227 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-run-netns\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.476400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476254 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-var-lib-kubelet\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.476400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476283 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm4gj\" (UniqueName: \"kubernetes.io/projected/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-kube-api-access-sm4gj\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.476400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476301 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:30.476400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476326 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-ovn-node-metrics-cert\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.476400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476352 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-run\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.476400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476376 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-etc-openvswitch\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.476400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476394 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-node-log\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476408 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-systemd\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476424 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snnh5\" (UniqueName: \"kubernetes.io/projected/67240493-12ad-4f5b-b73d-b1ecefbada09-kube-api-access-snnh5\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476440 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w77p7\" (UniqueName: \"kubernetes.io/projected/063fbbc6-c0dd-457e-b105-93c7874fb80a-kube-api-access-w77p7\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476466 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr5t8\" (UniqueName: \"kubernetes.io/projected/feb3ac88-148f-4921-9c03-dbe7f3d3da96-kube-api-access-vr5t8\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476482 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-slash\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476497 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-run-openvswitch\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476524 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-cnibin\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476548 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/063fbbc6-c0dd-457e-b105-93c7874fb80a-cni-binary-copy\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476579 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476600 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-etc-selinux\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476615 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-ovnkube-script-lib\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476638 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-system-cni-dir\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476658 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/538a3ccb-c26b-461c-96f5-152cd6eecb34-iptables-alerter-script\") pod \"iptables-alerter-6tccc\" (UID: \"538a3ccb-c26b-461c-96f5-152cd6eecb34\") " pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476677 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-cni-binary-copy\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476693 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.476829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476708 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-kubelet\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476730 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-sys\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476760 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-tuned\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476785 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-os-release\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476816 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46f8b6c0-73ab-4684-91c6-04e358626759-host\") pod \"node-ca-p67tg\" (UID: \"46f8b6c0-73ab-4684-91c6-04e358626759\") " pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476849 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4qgt\" (UniqueName: \"kubernetes.io/projected/46f8b6c0-73ab-4684-91c6-04e358626759-kube-api-access-d4qgt\") pod \"node-ca-p67tg\" (UID: \"46f8b6c0-73ab-4684-91c6-04e358626759\") " pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476872 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-systemd-units\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476890 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-cni-bin\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476906 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-ovnkube-config\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476957 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/669e056e-e810-451c-a00f-4e1c0a9d4c26-tmp-dir\") pod \"node-resolver-ctm9t\" (UID: \"669e056e-e810-451c-a00f-4e1c0a9d4c26\") " pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.476994 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5928g\" (UniqueName: \"kubernetes.io/projected/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-kube-api-access-5928g\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477018 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da9b854d-8ef1-4986-841d-25e45fd65bab-agent-certs\") pod \"konnectivity-agent-fqzkj\" (UID: \"da9b854d-8ef1-4986-841d-25e45fd65bab\") " pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477045 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-modprobe-d\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477068 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-sysctl-conf\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477090 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-var-lib-kubelet\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477113 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-daemon-config\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477142 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.477614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477167 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67240493-12ad-4f5b-b73d-b1ecefbada09-tmp\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477190 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477214 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-cni-dir\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477251 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-os-release\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477275 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-var-lib-cni-multus\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477311 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-registration-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477346 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/669e056e-e810-451c-a00f-4e1c0a9d4c26-hosts-file\") pod \"node-resolver-ctm9t\" (UID: \"669e056e-e810-451c-a00f-4e1c0a9d4c26\") " pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477385 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-sysctl-d\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477411 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-run-multus-certs\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477440 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-etc-kubernetes\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477466 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8mm4\" (UniqueName: \"kubernetes.io/projected/669e056e-e810-451c-a00f-4e1c0a9d4c26-kube-api-access-k8mm4\") pod \"node-resolver-ctm9t\" (UID: \"669e056e-e810-451c-a00f-4e1c0a9d4c26\") " pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477491 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-host\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477520 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-run-k8s-cni-cncf-io\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477550 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-system-cni-dir\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477577 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46f8b6c0-73ab-4684-91c6-04e358626759-serviceca\") pod \"node-ca-p67tg\" (UID: \"46f8b6c0-73ab-4684-91c6-04e358626759\") " pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477601 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-sys-fs\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477625 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/538a3ccb-c26b-461c-96f5-152cd6eecb34-host-slash\") pod \"iptables-alerter-6tccc\" (UID: \"538a3ccb-c26b-461c-96f5-152cd6eecb34\") " pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.478513 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477653 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-var-lib-openvswitch\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477679 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-cni-netd\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477716 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-kubernetes\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477744 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-var-lib-cni-bin\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477783 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-conf-dir\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477810 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-cnibin\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477837 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477863 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-sysconfig\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477900 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-hostroot\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477929 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-kubelet-config\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.477982 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj54d\" (UniqueName: \"kubernetes.io/projected/538a3ccb-c26b-461c-96f5-152cd6eecb34-kube-api-access-gj54d\") pod \"iptables-alerter-6tccc\" (UID: \"538a3ccb-c26b-461c-96f5-152cd6eecb34\") " pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478009 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-run-systemd\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478033 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-run-ovn\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478058 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478084 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-dbus\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478131 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-device-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478343 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-system-cni-dir\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.479388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478437 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-dbus\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478487 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlwff\" (UniqueName: \"kubernetes.io/projected/14379ef9-eea3-4198-815d-0903efebd55d-kube-api-access-jlwff\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478513 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-log-socket\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478542 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478572 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-socket-dir-parent\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478605 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478635 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-socket-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478706 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-run-netns\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478735 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-env-overrides\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478763 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da9b854d-8ef1-4986-841d-25e45fd65bab-konnectivity-ca\") pod \"konnectivity-agent-fqzkj\" (UID: \"da9b854d-8ef1-4986-841d-25e45fd65bab\") " pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.478932 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46f8b6c0-73ab-4684-91c6-04e358626759-serviceca\") pod \"node-ca-p67tg\" (UID: \"46f8b6c0-73ab-4684-91c6-04e358626759\") " pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.479081 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-cnibin\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.479344 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.479500 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-cni-binary-copy\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.479633 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.479701 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-kubelet-config\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.479766 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-os-release\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.480259 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.479807 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46f8b6c0-73ab-4684-91c6-04e358626759-host\") pod \"node-ca-p67tg\" (UID: \"46f8b6c0-73ab-4684-91c6-04e358626759\") " pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.481149 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.480105 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.481149 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.480209 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:30.481149 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.480323 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret podName:9c4bb811-9e79-4b09-84fe-a6e7debe4cee nodeName:}" failed. No retries permitted until 2026-07-10 19:07:30.980299727 +0000 UTC m=+2.996758090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret") pod "global-pull-secret-syncer-h42d9" (UID: "9c4bb811-9e79-4b09-84fe-a6e7debe4cee") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:30.486201 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.486173 2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 19:07:30.490816 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.490784 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm4gj\" (UniqueName: \"kubernetes.io/projected/668bc7ba-bf22-4bb2-ae0f-4c94bd912884-kube-api-access-sm4gj\") pod \"multus-additional-cni-plugins-k4fbp\" (UID: \"668bc7ba-bf22-4bb2-ae0f-4c94bd912884\") " pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.492694 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.492653 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4qgt\" (UniqueName: \"kubernetes.io/projected/46f8b6c0-73ab-4684-91c6-04e358626759-kube-api-access-d4qgt\") pod \"node-ca-p67tg\" (UID: \"46f8b6c0-73ab-4684-91c6-04e358626759\") " pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.516758 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.516692 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" event={"ID":"e39119c249ba3570b4866afa144c7646","Type":"ContainerStarted","Data":"d9dff77b280bce218dbd035d8cf5fb607d41ab43ba9abee84e72c4a4b18b169b"} Jul 10 19:07:30.518713 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.518682 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal" event={"ID":"24c9a12d95ac9c34c371cd5589b6402d","Type":"ContainerStarted","Data":"a2add955a3aad6f4024b0e616b5a948448366a6f351c924241a05e19ddfe6a4a"} Jul 10 19:07:30.579405 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579358 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-etc-selinux\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.579604 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579417 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-ovnkube-script-lib\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.579604 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579453 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-system-cni-dir\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.579604 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579472 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/538a3ccb-c26b-461c-96f5-152cd6eecb34-iptables-alerter-script\") pod \"iptables-alerter-6tccc\" (UID: \"538a3ccb-c26b-461c-96f5-152cd6eecb34\") " pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.579604 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579493 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-kubelet\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.579604 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579519 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-sys\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.579604 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579575 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-system-cni-dir\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.579604 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579601 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-sys\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.580021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579576 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-etc-selinux\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.580021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579682 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-tuned\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.580021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579730 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-systemd-units\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.580021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579735 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-kubelet\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.580021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579755 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-cni-bin\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.580021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579783 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-ovnkube-config\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.580021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579895 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/669e056e-e810-451c-a00f-4e1c0a9d4c26-tmp-dir\") pod \"node-resolver-ctm9t\" (UID: \"669e056e-e810-451c-a00f-4e1c0a9d4c26\") " pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.580021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579946 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5928g\" (UniqueName: \"kubernetes.io/projected/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-kube-api-access-5928g\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.580021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579963 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-systemd-units\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.580476 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.580072 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-cni-bin\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.580476 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.580305 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/538a3ccb-c26b-461c-96f5-152cd6eecb34-iptables-alerter-script\") pod \"iptables-alerter-6tccc\" (UID: \"538a3ccb-c26b-461c-96f5-152cd6eecb34\") " pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.580476 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.580364 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-ovnkube-script-lib\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.580724 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.580678 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/669e056e-e810-451c-a00f-4e1c0a9d4c26-tmp-dir\") pod \"node-resolver-ctm9t\" (UID: \"669e056e-e810-451c-a00f-4e1c0a9d4c26\") " pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.580789 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.579972 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da9b854d-8ef1-4986-841d-25e45fd65bab-agent-certs\") pod \"konnectivity-agent-fqzkj\" (UID: \"da9b854d-8ef1-4986-841d-25e45fd65bab\") " pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:30.584385 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584333 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-ovnkube-config\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.584551 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584524 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-modprobe-d\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.584641 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584596 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-sysctl-conf\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.584641 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584636 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-var-lib-kubelet\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.584753 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584673 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-daemon-config\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.584753 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584709 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67240493-12ad-4f5b-b73d-b1ecefbada09-tmp\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.584753 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584739 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:30.584948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584801 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-cni-dir\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.584948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584834 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-os-release\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.584948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584919 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-var-lib-cni-multus\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.585101 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584950 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-registration-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.585101 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.584983 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/669e056e-e810-451c-a00f-4e1c0a9d4c26-hosts-file\") pod \"node-resolver-ctm9t\" (UID: \"669e056e-e810-451c-a00f-4e1c0a9d4c26\") " pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.585101 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585014 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-sysctl-d\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.585101 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585060 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-run-multus-certs\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.585226 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585114 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-etc-kubernetes\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.585226 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585143 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8mm4\" (UniqueName: \"kubernetes.io/projected/669e056e-e810-451c-a00f-4e1c0a9d4c26-kube-api-access-k8mm4\") pod \"node-resolver-ctm9t\" (UID: \"669e056e-e810-451c-a00f-4e1c0a9d4c26\") " pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.585226 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585174 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-host\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.585226 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585207 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-run-k8s-cni-cncf-io\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.585348 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585243 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-sys-fs\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.585348 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585277 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/538a3ccb-c26b-461c-96f5-152cd6eecb34-host-slash\") pod \"iptables-alerter-6tccc\" (UID: \"538a3ccb-c26b-461c-96f5-152cd6eecb34\") " pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.585348 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585315 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-var-lib-openvswitch\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.585437 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585351 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-sysctl-conf\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.585437 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585381 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-cni-netd\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.585437 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585419 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-kubernetes\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.585559 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585465 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-var-lib-cni-bin\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.585559 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585499 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-conf-dir\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.585559 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585506 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-sysctl-d\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.585559 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-sysconfig\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.585678 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585576 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-hostroot\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.585678 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585583 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-run-multus-certs\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.585678 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585606 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj54d\" (UniqueName: \"kubernetes.io/projected/538a3ccb-c26b-461c-96f5-152cd6eecb34-kube-api-access-gj54d\") pod \"iptables-alerter-6tccc\" (UID: \"538a3ccb-c26b-461c-96f5-152cd6eecb34\") " pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.585678 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585637 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-run-systemd\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.585678 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585636 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-etc-kubernetes\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.585678 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585658 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-run-ovn\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.585866 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585682 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.585866 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585719 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-device-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.585866 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585744 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlwff\" (UniqueName: \"kubernetes.io/projected/14379ef9-eea3-4198-815d-0903efebd55d-kube-api-access-jlwff\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.585866 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585768 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-log-socket\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.585866 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585791 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.585866 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585818 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-socket-dir-parent\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585858 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-socket-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585898 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-run-netns\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585919 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-env-overrides\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585939 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da9b854d-8ef1-4986-841d-25e45fd65bab-konnectivity-ca\") pod \"konnectivity-agent-fqzkj\" (UID: \"da9b854d-8ef1-4986-841d-25e45fd65bab\") " pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.585988 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-lib-modules\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586019 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-run-netns\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586056 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-var-lib-kubelet\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586070 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-host\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586096 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586133 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-ovn-node-metrics-cert\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586130 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-run-k8s-cni-cncf-io\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586170 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-run\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586221 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-sys-fs\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586245 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-etc-openvswitch\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586282 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/538a3ccb-c26b-461c-96f5-152cd6eecb34-host-slash\") pod \"iptables-alerter-6tccc\" (UID: \"538a3ccb-c26b-461c-96f5-152cd6eecb34\") " pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586283 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-node-log\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586332 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-node-log\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586341 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-systemd\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586407 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-var-lib-kubelet\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586440 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snnh5\" (UniqueName: \"kubernetes.io/projected/67240493-12ad-4f5b-b73d-b1ecefbada09-kube-api-access-snnh5\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586433 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-var-lib-openvswitch\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586466 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77p7\" (UniqueName: \"kubernetes.io/projected/063fbbc6-c0dd-457e-b105-93c7874fb80a-kube-api-access-w77p7\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586498 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5t8\" (UniqueName: \"kubernetes.io/projected/feb3ac88-148f-4921-9c03-dbe7f3d3da96-kube-api-access-vr5t8\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586510 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-cni-netd\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586533 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-slash\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586564 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-run-openvswitch\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586597 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-cnibin\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586632 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/063fbbc6-c0dd-457e-b105-93c7874fb80a-cni-binary-copy\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.586727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586624 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-modprobe-d\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.587675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586682 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.587675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586733 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-kubernetes\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.587675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586802 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.587675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586810 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-systemd\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.587675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586877 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-var-lib-cni-bin\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.587675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.586977 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-socket-dir-parent\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.587675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.587412 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-daemon-config\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.587675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.587487 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-run\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.587675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.587530 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-etc-openvswitch\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.588198 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588174 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-slash\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.588269 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588156 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da9b854d-8ef1-4986-841d-25e45fd65bab-agent-certs\") pod \"konnectivity-agent-fqzkj\" (UID: \"da9b854d-8ef1-4986-841d-25e45fd65bab\") " pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:30.588329 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588274 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-run-openvswitch\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.588396 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588349 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-cnibin\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.588396 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588353 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-tuned\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.588544 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588523 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-lib-modules\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.588717 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588698 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-socket-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.588777 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588763 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-run-netns\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.588883 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588862 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/063fbbc6-c0dd-457e-b105-93c7874fb80a-cni-binary-copy\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.588983 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.588963 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-var-lib-kubelet\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.589057 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589027 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-run-netns\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.589271 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589249 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-env-overrides\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.589369 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589351 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-host-var-lib-cni-multus\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.589613 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589592 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da9b854d-8ef1-4986-841d-25e45fd65bab-konnectivity-ca\") pod \"konnectivity-agent-fqzkj\" (UID: \"da9b854d-8ef1-4986-841d-25e45fd65bab\") " pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:30.589707 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589690 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/669e056e-e810-451c-a00f-4e1c0a9d4c26-hosts-file\") pod \"node-resolver-ctm9t\" (UID: \"669e056e-e810-451c-a00f-4e1c0a9d4c26\") " pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.589764 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589737 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-cni-dir\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.589823 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589778 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-registration-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.589877 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589821 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-os-release\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.589932 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589873 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-multus-conf-dir\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.589932 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.589880 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:30.590067 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.589932 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/67240493-12ad-4f5b-b73d-b1ecefbada09-etc-sysconfig\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.590067 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.589974 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs podName:feb3ac88-148f-4921-9c03-dbe7f3d3da96 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:31.089931134 +0000 UTC m=+3.106389489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs") pod "network-metrics-daemon-7ng4r" (UID: "feb3ac88-148f-4921-9c03-dbe7f3d3da96") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:30.590067 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.590059 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.590240 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.590223 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-run-systemd\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.590504 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.590438 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-run-ovn\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.590570 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.590539 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14379ef9-eea3-4198-815d-0903efebd55d-device-dir\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.590632 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.590603 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-log-socket\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.590687 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.590659 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.591875 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.591843 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/063fbbc6-c0dd-457e-b105-93c7874fb80a-hostroot\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.595120 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.594779 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:07:30.595120 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.594808 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:07:30.595120 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.594832 2556 projected.go:196] Error preparing data for projected volume kube-api-access-btjgd for pod openshift-network-diagnostics/network-check-target-cn5r4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:30.595120 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.594954 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd podName:bf3e3721-db7c-4586-ba94-72732ce64494 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:31.094914268 +0000 UTC m=+3.111372644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-btjgd" (UniqueName: "kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd") pod "network-check-target-cn5r4" (UID: "bf3e3721-db7c-4586-ba94-72732ce64494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:30.595564 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.595538 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-ovn-node-metrics-cert\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.600009 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.599978 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5928g\" (UniqueName: \"kubernetes.io/projected/14e7d49a-c59a-41c4-b0e7-357cc681a3a8-kube-api-access-5928g\") pod \"ovnkube-node-29hgv\" (UID: \"14e7d49a-c59a-41c4-b0e7-357cc681a3a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.600215 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.600189 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlwff\" (UniqueName: \"kubernetes.io/projected/14379ef9-eea3-4198-815d-0903efebd55d-kube-api-access-jlwff\") pod \"aws-ebs-csi-driver-node-mm9bk\" (UID: \"14379ef9-eea3-4198-815d-0903efebd55d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.607841 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.607806 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snnh5\" (UniqueName: \"kubernetes.io/projected/67240493-12ad-4f5b-b73d-b1ecefbada09-kube-api-access-snnh5\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.607999 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.607842 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr5t8\" (UniqueName: \"kubernetes.io/projected/feb3ac88-148f-4921-9c03-dbe7f3d3da96-kube-api-access-vr5t8\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:30.607999 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.607818 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w77p7\" (UniqueName: \"kubernetes.io/projected/063fbbc6-c0dd-457e-b105-93c7874fb80a-kube-api-access-w77p7\") pod \"multus-rcvkj\" (UID: \"063fbbc6-c0dd-457e-b105-93c7874fb80a\") " pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.607999 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.607980 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67240493-12ad-4f5b-b73d-b1ecefbada09-tmp\") pod \"tuned-dk4vx\" (UID: \"67240493-12ad-4f5b-b73d-b1ecefbada09\") " pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.608174 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.608034 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj54d\" (UniqueName: \"kubernetes.io/projected/538a3ccb-c26b-461c-96f5-152cd6eecb34-kube-api-access-gj54d\") pod \"iptables-alerter-6tccc\" (UID: \"538a3ccb-c26b-461c-96f5-152cd6eecb34\") " pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.608294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.608263 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8mm4\" (UniqueName: \"kubernetes.io/projected/669e056e-e810-451c-a00f-4e1c0a9d4c26-kube-api-access-k8mm4\") pod \"node-resolver-ctm9t\" (UID: \"669e056e-e810-451c-a00f-4e1c0a9d4c26\") " pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.679815 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.679765 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" Jul 10 19:07:30.693240 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:30.693201 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod668bc7ba_bf22_4bb2_ae0f_4c94bd912884.slice/crio-d880047a4a5aa4d0ccbdd061ee55c669915f59496ffe2fb45248ecc011133c7b WatchSource:0}: Error finding container d880047a4a5aa4d0ccbdd061ee55c669915f59496ffe2fb45248ecc011133c7b: Status 404 returned error can't find the container with id d880047a4a5aa4d0ccbdd061ee55c669915f59496ffe2fb45248ecc011133c7b Jul 10 19:07:30.695199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.694928 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p67tg" Jul 10 19:07:30.705527 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:30.705491 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46f8b6c0_73ab_4684_91c6_04e358626759.slice/crio-f753ad1070d9f3b571199deb08a5bef6c176336b00e99817dddeead1d47257dc WatchSource:0}: Error finding container f753ad1070d9f3b571199deb08a5bef6c176336b00e99817dddeead1d47257dc: Status 404 returned error can't find the container with id f753ad1070d9f3b571199deb08a5bef6c176336b00e99817dddeead1d47257dc Jul 10 19:07:30.708473 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.708452 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:30.716324 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.716288 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" Jul 10 19:07:30.725577 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.725548 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" Jul 10 19:07:30.726414 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:30.726385 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14379ef9_eea3_4198_815d_0903efebd55d.slice/crio-28eaaced10a40619cc5ebd798e0fe3265b0ad8874dc279be13945b9f7b6b1bd6 WatchSource:0}: Error finding container 28eaaced10a40619cc5ebd798e0fe3265b0ad8874dc279be13945b9f7b6b1bd6: Status 404 returned error can't find the container with id 28eaaced10a40619cc5ebd798e0fe3265b0ad8874dc279be13945b9f7b6b1bd6 Jul 10 19:07:30.736094 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:30.736060 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67240493_12ad_4f5b_b73d_b1ecefbada09.slice/crio-e4806513499094cbb5ba710996c9b12e3fef6a8b504313e306bf0d97b444a1a0 WatchSource:0}: Error finding container e4806513499094cbb5ba710996c9b12e3fef6a8b504313e306bf0d97b444a1a0: Status 404 returned error can't find the container with id e4806513499094cbb5ba710996c9b12e3fef6a8b504313e306bf0d97b444a1a0 Jul 10 19:07:30.744742 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.744689 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6tccc" Jul 10 19:07:30.754964 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.754703 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:30.755110 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:30.755038 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538a3ccb_c26b_461c_96f5_152cd6eecb34.slice/crio-79928966fc36724801cce03b7375b1aa569d79c383ac4340a19d7047cf0946a7 WatchSource:0}: Error finding container 79928966fc36724801cce03b7375b1aa569d79c383ac4340a19d7047cf0946a7: Status 404 returned error can't find the container with id 79928966fc36724801cce03b7375b1aa569d79c383ac4340a19d7047cf0946a7 Jul 10 19:07:30.762825 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.762799 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rcvkj" Jul 10 19:07:30.767937 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:30.767900 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e7d49a_c59a_41c4_b0e7_357cc681a3a8.slice/crio-a64111af18eefe0b3200e91b8c02be6acd3b85c97ea10a993b7a5303d4aba2e6 WatchSource:0}: Error finding container a64111af18eefe0b3200e91b8c02be6acd3b85c97ea10a993b7a5303d4aba2e6: Status 404 returned error can't find the container with id a64111af18eefe0b3200e91b8c02be6acd3b85c97ea10a993b7a5303d4aba2e6 Jul 10 19:07:30.770653 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.770388 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ctm9t" Jul 10 19:07:30.777037 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:30.776748 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063fbbc6_c0dd_457e_b105_93c7874fb80a.slice/crio-ad402b47c1c4bfd253e53df3a88b54b5391ef3172a13f4d05291673a5e50f283 WatchSource:0}: Error finding container ad402b47c1c4bfd253e53df3a88b54b5391ef3172a13f4d05291673a5e50f283: Status 404 returned error can't find the container with id ad402b47c1c4bfd253e53df3a88b54b5391ef3172a13f4d05291673a5e50f283 Jul 10 19:07:30.784581 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:07:30.784544 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod669e056e_e810_451c_a00f_4e1c0a9d4c26.slice/crio-2e219b40a95e1e1778577be03217cc6ba3d72936437aec4d6b824f9cb3483f47 WatchSource:0}: Error finding container 2e219b40a95e1e1778577be03217cc6ba3d72936437aec4d6b824f9cb3483f47: Status 404 returned error can't find the container with id 2e219b40a95e1e1778577be03217cc6ba3d72936437aec4d6b824f9cb3483f47 Jul 10 19:07:30.835412 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.835370 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:07:30.927697 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.927608 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 19:07:30.990119 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:30.990083 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:30.990323 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.990267 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:30.990392 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:30.990357 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret podName:9c4bb811-9e79-4b09-84fe-a6e7debe4cee nodeName:}" failed. No retries permitted until 2026-07-10 19:07:31.990333244 +0000 UTC m=+4.006791598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret") pod "global-pull-secret-syncer-h42d9" (UID: "9c4bb811-9e79-4b09-84fe-a6e7debe4cee") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:31.091279 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.091238 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:31.091493 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:31.091427 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:31.091566 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:31.091516 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs podName:feb3ac88-148f-4921-9c03-dbe7f3d3da96 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:32.091493045 +0000 UTC m=+4.107951408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs") pod "network-metrics-daemon-7ng4r" (UID: "feb3ac88-148f-4921-9c03-dbe7f3d3da96") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:31.192785 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.192682 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:31.192998 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:31.192907 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:07:31.192998 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:31.192938 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:07:31.192998 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:31.192949 2556 projected.go:196] Error preparing data for projected volume kube-api-access-btjgd for pod openshift-network-diagnostics/network-check-target-cn5r4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:31.193170 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:31.193015 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd podName:bf3e3721-db7c-4586-ba94-72732ce64494 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:32.193000218 +0000 UTC m=+4.209458572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-btjgd" (UniqueName: "kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd") pod "network-check-target-cn5r4" (UID: "bf3e3721-db7c-4586-ba94-72732ce64494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:31.414166 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.414044 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 19:02:31 +0000 UTC" deadline="2028-03-11 21:16:10.794648456 +0000 UTC" Jul 10 19:07:31.414166 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.414102 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14642h8m39.380551442s" Jul 10 19:07:31.511440 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.511307 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:31.511624 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:31.511486 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:31.534464 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.534417 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ctm9t" event={"ID":"669e056e-e810-451c-a00f-4e1c0a9d4c26","Type":"ContainerStarted","Data":"2e219b40a95e1e1778577be03217cc6ba3d72936437aec4d6b824f9cb3483f47"} Jul 10 19:07:31.538425 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.538385 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerStarted","Data":"a64111af18eefe0b3200e91b8c02be6acd3b85c97ea10a993b7a5303d4aba2e6"} Jul 10 19:07:31.540212 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.540178 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" event={"ID":"67240493-12ad-4f5b-b73d-b1ecefbada09","Type":"ContainerStarted","Data":"e4806513499094cbb5ba710996c9b12e3fef6a8b504313e306bf0d97b444a1a0"} Jul 10 19:07:31.542238 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.542205 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fqzkj" event={"ID":"da9b854d-8ef1-4986-841d-25e45fd65bab","Type":"ContainerStarted","Data":"c89bd05dc2bf6c1db9285f8126cc04fe1009e4fab4674f57e0e562f0d519a450"} Jul 10 19:07:31.548443 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.548401 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p67tg" event={"ID":"46f8b6c0-73ab-4684-91c6-04e358626759","Type":"ContainerStarted","Data":"f753ad1070d9f3b571199deb08a5bef6c176336b00e99817dddeead1d47257dc"} Jul 10 19:07:31.552414 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.552378 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rcvkj" event={"ID":"063fbbc6-c0dd-457e-b105-93c7874fb80a","Type":"ContainerStarted","Data":"ad402b47c1c4bfd253e53df3a88b54b5391ef3172a13f4d05291673a5e50f283"} Jul 10 19:07:31.556448 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.556408 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6tccc" event={"ID":"538a3ccb-c26b-461c-96f5-152cd6eecb34","Type":"ContainerStarted","Data":"79928966fc36724801cce03b7375b1aa569d79c383ac4340a19d7047cf0946a7"} Jul 10 19:07:31.558611 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.558578 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" event={"ID":"14379ef9-eea3-4198-815d-0903efebd55d","Type":"ContainerStarted","Data":"28eaaced10a40619cc5ebd798e0fe3265b0ad8874dc279be13945b9f7b6b1bd6"} Jul 10 19:07:31.561294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:31.561261 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" event={"ID":"668bc7ba-bf22-4bb2-ae0f-4c94bd912884","Type":"ContainerStarted","Data":"d880047a4a5aa4d0ccbdd061ee55c669915f59496ffe2fb45248ecc011133c7b"} Jul 10 19:07:32.000962 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:32.000811 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:32.001127 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.001026 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:32.001127 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.001115 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret podName:9c4bb811-9e79-4b09-84fe-a6e7debe4cee nodeName:}" failed. No retries permitted until 2026-07-10 19:07:34.001093231 +0000 UTC m=+6.017551593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret") pod "global-pull-secret-syncer-h42d9" (UID: "9c4bb811-9e79-4b09-84fe-a6e7debe4cee") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:32.101337 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:32.101286 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:32.101569 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.101548 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:32.101666 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.101623 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs podName:feb3ac88-148f-4921-9c03-dbe7f3d3da96 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:34.101602835 +0000 UTC m=+6.118061198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs") pod "network-metrics-daemon-7ng4r" (UID: "feb3ac88-148f-4921-9c03-dbe7f3d3da96") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:32.202094 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:32.202031 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:32.202300 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.202286 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:07:32.202379 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.202312 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:07:32.202379 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.202325 2556 projected.go:196] Error preparing data for projected volume kube-api-access-btjgd for pod openshift-network-diagnostics/network-check-target-cn5r4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:32.202487 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.202390 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd podName:bf3e3721-db7c-4586-ba94-72732ce64494 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:34.202369462 +0000 UTC m=+6.218827816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-btjgd" (UniqueName: "kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd") pod "network-check-target-cn5r4" (UID: "bf3e3721-db7c-4586-ba94-72732ce64494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:32.415118 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:32.415019 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 19:02:31 +0000 UTC" deadline="2027-12-10 06:12:35.138262332 +0000 UTC" Jul 10 19:07:32.415118 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:32.415069 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12419h5m2.72319931s" Jul 10 19:07:32.513136 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:32.513099 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:32.513320 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.513260 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:32.513453 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:32.513433 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:32.513558 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:32.513537 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:33.513177 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:33.512410 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:33.513177 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:33.512561 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:34.016385 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:34.016338 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:34.016659 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.016634 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:34.016755 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.016728 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret podName:9c4bb811-9e79-4b09-84fe-a6e7debe4cee nodeName:}" failed. No retries permitted until 2026-07-10 19:07:38.016704877 +0000 UTC m=+10.033163249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret") pod "global-pull-secret-syncer-h42d9" (UID: "9c4bb811-9e79-4b09-84fe-a6e7debe4cee") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:34.117608 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:34.117562 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:34.117818 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.117798 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:34.117943 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.117877 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs podName:feb3ac88-148f-4921-9c03-dbe7f3d3da96 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:38.117855834 +0000 UTC m=+10.134314210 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs") pod "network-metrics-daemon-7ng4r" (UID: "feb3ac88-148f-4921-9c03-dbe7f3d3da96") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:34.219054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:34.219000 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:34.219227 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.219176 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:07:34.219227 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.219203 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:07:34.219227 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.219216 2556 projected.go:196] Error preparing data for projected volume kube-api-access-btjgd for pod openshift-network-diagnostics/network-check-target-cn5r4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:34.219433 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.219282 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd podName:bf3e3721-db7c-4586-ba94-72732ce64494 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:38.219262864 +0000 UTC m=+10.235721233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-btjgd" (UniqueName: "kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd") pod "network-check-target-cn5r4" (UID: "bf3e3721-db7c-4586-ba94-72732ce64494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:34.512096 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:34.512009 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:34.512278 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.512152 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:34.512623 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:34.512604 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:34.512743 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:34.512721 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:35.511559 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:35.511517 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:35.512180 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:35.511691 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:36.513809 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:36.513775 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:36.514223 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:36.513894 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:36.514353 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:36.514326 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:36.514470 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:36.514445 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:37.511453 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:37.511358 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:37.511620 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:37.511516 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:38.049722 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:38.049681 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:38.050250 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.049874 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:38.050250 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.049986 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret podName:9c4bb811-9e79-4b09-84fe-a6e7debe4cee nodeName:}" failed. No retries permitted until 2026-07-10 19:07:46.04995859 +0000 UTC m=+18.066416946 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret") pod "global-pull-secret-syncer-h42d9" (UID: "9c4bb811-9e79-4b09-84fe-a6e7debe4cee") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:38.150549 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:38.150514 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:38.150742 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.150647 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:38.150742 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.150723 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs podName:feb3ac88-148f-4921-9c03-dbe7f3d3da96 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:46.150704809 +0000 UTC m=+18.167163160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs") pod "network-metrics-daemon-7ng4r" (UID: "feb3ac88-148f-4921-9c03-dbe7f3d3da96") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:38.251802 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:38.251761 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:38.252018 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.251967 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:07:38.252018 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.251994 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:07:38.252018 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.252007 2556 projected.go:196] Error preparing data for projected volume kube-api-access-btjgd for pod openshift-network-diagnostics/network-check-target-cn5r4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:38.252198 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.252070 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd podName:bf3e3721-db7c-4586-ba94-72732ce64494 nodeName:}" failed. No retries permitted until 2026-07-10 19:07:46.252050041 +0000 UTC m=+18.268508405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-btjgd" (UniqueName: "kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd") pod "network-check-target-cn5r4" (UID: "bf3e3721-db7c-4586-ba94-72732ce64494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:38.512429 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:38.512232 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:38.512429 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.512413 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:38.512811 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:38.512779 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:38.512968 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:38.512884 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:39.511256 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:39.511219 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:39.511743 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:39.511354 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:40.513097 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:40.513060 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:40.513539 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:40.513183 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:40.513539 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:40.513493 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:40.513622 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:40.513563 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:41.511502 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:41.511048 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:41.511502 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:41.511188 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:42.511453 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:42.511381 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:42.513153 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:42.513111 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:42.513639 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:42.513612 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:42.513803 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:42.513778 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:43.511432 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:43.511398 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:43.511613 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:43.511527 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:44.511024 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:44.510987 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:44.511222 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:44.511122 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:44.511222 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:44.511180 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:44.511325 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:44.511299 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:45.511048 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:45.511007 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:45.511459 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:45.511154 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:46.114434 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:46.114383 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:46.114619 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.114565 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:46.114683 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.114630 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret podName:9c4bb811-9e79-4b09-84fe-a6e7debe4cee nodeName:}" failed. No retries permitted until 2026-07-10 19:08:02.114612708 +0000 UTC m=+34.131071057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret") pod "global-pull-secret-syncer-h42d9" (UID: "9c4bb811-9e79-4b09-84fe-a6e7debe4cee") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:07:46.214864 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:46.214823 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:46.215074 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.215010 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:46.215074 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.215075 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs podName:feb3ac88-148f-4921-9c03-dbe7f3d3da96 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:02.21505999 +0000 UTC m=+34.231518348 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs") pod "network-metrics-daemon-7ng4r" (UID: "feb3ac88-148f-4921-9c03-dbe7f3d3da96") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:07:46.315557 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:46.315516 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:46.315729 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.315671 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:07:46.315729 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.315696 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:07:46.315729 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.315709 2556 projected.go:196] Error preparing data for projected volume kube-api-access-btjgd for pod openshift-network-diagnostics/network-check-target-cn5r4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:46.315863 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.315780 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd podName:bf3e3721-db7c-4586-ba94-72732ce64494 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:02.31575991 +0000 UTC m=+34.332218265 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-btjgd" (UniqueName: "kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd") pod "network-check-target-cn5r4" (UID: "bf3e3721-db7c-4586-ba94-72732ce64494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:07:46.512813 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:46.510934 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:46.512813 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:46.510991 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:46.512813 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.511068 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:46.512813 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:46.511205 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:47.511165 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:47.511126 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:47.511352 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:47.511257 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:48.512366 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:48.512326 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:48.512829 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:48.512468 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:48.512829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:48.512508 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:48.512829 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:48.512647 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:49.511713 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:49.511667 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:49.512046 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:49.511813 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:50.512572 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:50.512014 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:50.512572 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:50.512114 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:50.512572 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:50.512479 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:50.513538 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:50.513484 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:50.608558 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:50.608282 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rcvkj" event={"ID":"063fbbc6-c0dd-457e-b105-93c7874fb80a","Type":"ContainerStarted","Data":"fd78ccf0511de998cc37862a90a26cdbb1608182dc39277336a6cd0781000b76"} Jul 10 19:07:50.609774 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:50.609739 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal" event={"ID":"24c9a12d95ac9c34c371cd5589b6402d","Type":"ContainerStarted","Data":"3d2f4b00860df4aaf099ca505b598258bf9d5ef1d0f9a198109e450318a7eca3"} Jul 10 19:07:50.613017 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:50.612985 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" event={"ID":"67240493-12ad-4f5b-b73d-b1ecefbada09","Type":"ContainerStarted","Data":"3fba49078934d6f7fedb7211144b333cfdd2c50d7cee76a027e9cf573f1ea84f"} Jul 10 19:07:50.627309 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:50.627256 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-73.ec2.internal" podStartSLOduration=20.627242512 podStartE2EDuration="20.627242512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:50.626759918 +0000 UTC m=+22.643218292" watchObservedRunningTime="2026-07-10 19:07:50.627242512 +0000 UTC m=+22.643700941" Jul 10 19:07:50.643305 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:50.643258 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dk4vx" podStartSLOduration=2.010434921 podStartE2EDuration="21.643241457s" totalImagesPullingTime="19.632806536s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:50.643163189 +0000 UTC m=+22.659621560" watchObservedRunningTime="2026-07-10 19:07:50.643241457 +0000 UTC m=+22.659699842" Jul 10 19:07:51.511750 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.511710 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:51.511967 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:51.511927 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:51.617294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.617032 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ctm9t" event={"ID":"669e056e-e810-451c-a00f-4e1c0a9d4c26","Type":"ContainerStarted","Data":"77cef3234ebf9bedaad35fb58edd6d91352e8dad53b3e7ab8b75cf36aad3c2d9"} Jul 10 19:07:51.619911 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.619862 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:07:51.620235 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.620213 2556 generic.go:356] "Generic (PLEG): container finished" podID="14e7d49a-c59a-41c4-b0e7-357cc681a3a8" containerID="e9fac02fc201575f37d7b0a793bef2bdc0c5223e15ebcb35ff382ee27926d4c0" exitCode=1 Jul 10 19:07:51.620324 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.620289 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerStarted","Data":"5583e4775da01e66009d669e7dc3893aeac1307685ac5545f933f38b8ca21882"} Jul 10 19:07:51.620391 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.620373 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerStarted","Data":"5dbe3a1a366ac3cbe2e8d8048487345a65f997b743636ce513e88b24d88ed4a9"} Jul 10 19:07:51.620438 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.620419 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerStarted","Data":"9c68f57f89b444fcbb8e4be8884f05ea602d16cc8d9c64604003c16ae537b509"} Jul 10 19:07:51.620473 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.620459 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerStarted","Data":"28d590f7c0e689942eab094867957f55bb664d02c28ba6ffa3c69c4d06fbc1ba"} Jul 10 19:07:51.620518 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.620494 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerStarted","Data":"e2befd454c5c9aedc762e00ebd953de0d09b404dbc31dac78af6d6a773f02efb"} Jul 10 19:07:51.620557 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.620539 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerDied","Data":"e9fac02fc201575f37d7b0a793bef2bdc0c5223e15ebcb35ff382ee27926d4c0"} Jul 10 19:07:51.621731 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.621705 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fqzkj" event={"ID":"da9b854d-8ef1-4986-841d-25e45fd65bab","Type":"ContainerStarted","Data":"01caa93407b11fbd643cfebf5c292043b52580253b43b81abc4563bd919e82a0"} Jul 10 19:07:51.623343 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.623317 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p67tg" event={"ID":"46f8b6c0-73ab-4684-91c6-04e358626759","Type":"ContainerStarted","Data":"2b894e0646bdcfcc795d7bfef2648a59e7c52fbb993eaf52409a1be9e36e03b0"} Jul 10 19:07:51.624787 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.624764 2556 generic.go:356] "Generic (PLEG): container finished" podID="e39119c249ba3570b4866afa144c7646" containerID="f56a7e2d89616e8e9a11d9a9307ecc15458798b86823070f177ae31e1f7e363a" exitCode=0 Jul 10 19:07:51.624908 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.624833 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" event={"ID":"e39119c249ba3570b4866afa144c7646","Type":"ContainerDied","Data":"f56a7e2d89616e8e9a11d9a9307ecc15458798b86823070f177ae31e1f7e363a"} Jul 10 19:07:51.626242 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.626219 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" event={"ID":"14379ef9-eea3-4198-815d-0903efebd55d","Type":"ContainerStarted","Data":"09e417d3ee5463e056d047b1d0c8545780f0a7aaff003ff7e15733f64ea75f21"} Jul 10 19:07:51.627565 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.627541 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" event={"ID":"668bc7ba-bf22-4bb2-ae0f-4c94bd912884","Type":"ContainerStarted","Data":"5e282fd8c254067027ce432eb66e1a962104f9ec591af2b8307b77a0c9ed48f8"} Jul 10 19:07:51.630773 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.630734 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ctm9t" podStartSLOduration=3.046478718 podStartE2EDuration="22.630721319s" totalImagesPullingTime="19.584242601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:51.630079767 +0000 UTC m=+23.646538139" watchObservedRunningTime="2026-07-10 19:07:51.630721319 +0000 UTC m=+23.647179686" Jul 10 19:07:51.680087 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.680038 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fqzkj" podStartSLOduration=3.034862196 podStartE2EDuration="22.680020397s" totalImagesPullingTime="19.645158201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:51.664337633 +0000 UTC m=+23.680796005" watchObservedRunningTime="2026-07-10 19:07:51.680020397 +0000 UTC m=+23.696478768" Jul 10 19:07:51.680311 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.680291 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rcvkj" podStartSLOduration=3.041734444 podStartE2EDuration="22.680282969s" totalImagesPullingTime="19.638548525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:51.679997265 +0000 UTC m=+23.696455635" watchObservedRunningTime="2026-07-10 19:07:51.680282969 +0000 UTC m=+23.696741340" Jul 10 19:07:51.695495 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:51.695450 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p67tg" podStartSLOduration=11.556425215 podStartE2EDuration="22.695438265s" totalImagesPullingTime="11.13901305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:51.69503577 +0000 UTC m=+23.711494142" watchObservedRunningTime="2026-07-10 19:07:51.695438265 +0000 UTC m=+23.711896636" Jul 10 19:07:52.511655 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:52.511612 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:52.511851 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:52.511727 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:52.511851 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:52.511788 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:52.511998 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:52.511931 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:52.631052 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:52.630987 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6tccc" event={"ID":"538a3ccb-c26b-461c-96f5-152cd6eecb34","Type":"ContainerStarted","Data":"dd4a5f730a0ad47d9b85111c452b5b8401aaf2d3872c360ab7d50292ad31c85b"} Jul 10 19:07:52.632381 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:52.632354 2556 generic.go:356] "Generic (PLEG): container finished" podID="668bc7ba-bf22-4bb2-ae0f-4c94bd912884" containerID="5e282fd8c254067027ce432eb66e1a962104f9ec591af2b8307b77a0c9ed48f8" exitCode=0 Jul 10 19:07:52.632508 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:52.632449 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" event={"ID":"668bc7ba-bf22-4bb2-ae0f-4c94bd912884","Type":"ContainerDied","Data":"5e282fd8c254067027ce432eb66e1a962104f9ec591af2b8307b77a0c9ed48f8"} Jul 10 19:07:52.645254 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:52.645196 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6tccc" podStartSLOduration=4.009534573 podStartE2EDuration="23.645181009s" totalImagesPullingTime="19.635646436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:52.644745048 +0000 UTC m=+24.661203418" watchObservedRunningTime="2026-07-10 19:07:52.645181009 +0000 UTC m=+24.661639379" Jul 10 19:07:52.940055 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:52.939998 2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 19:07:53.438403 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:53.438295 2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T19:07:52.940018148Z","UUID":"6e11d5b8-a581-4575-9876-99df83fbd46f","Handler":null,"Name":"","Endpoint":""} Jul 10 19:07:53.439984 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:53.439956 2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 19:07:53.440130 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:53.440005 2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 19:07:53.512169 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:53.512136 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:53.512354 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:53.512276 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:53.639607 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:53.639532 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:07:53.640095 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:53.639988 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerStarted","Data":"4c0eaf7e388628fca5a34e7acfb248a742205bea11d161063d3c69ff5392d206"} Jul 10 19:07:53.642483 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:53.642448 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" event={"ID":"e39119c249ba3570b4866afa144c7646","Type":"ContainerStarted","Data":"3287a6f7dc1982a6f619d363ad5c20b94c3b1c1fc282972a62ce77f909f5b95d"} Jul 10 19:07:53.644743 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:53.644718 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" event={"ID":"14379ef9-eea3-4198-815d-0903efebd55d","Type":"ContainerStarted","Data":"9905c58cdf2f63a2ef4476c59c7acfa0e3deed767dc5840889a6c216d135a2c1"} Jul 10 19:07:53.667619 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:53.667549 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-73.ec2.internal" podStartSLOduration=23.667527898 podStartE2EDuration="23.667527898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:53.667019627 +0000 UTC m=+25.683477997" watchObservedRunningTime="2026-07-10 19:07:53.667527898 +0000 UTC m=+25.683986271" Jul 10 19:07:54.511822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:54.511781 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:54.512009 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:54.511905 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:54.512009 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:54.511996 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:54.512099 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:54.512084 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:54.649246 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:54.649044 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" event={"ID":"14379ef9-eea3-4198-815d-0903efebd55d","Type":"ContainerStarted","Data":"d8d2a21b2eb52c2770d99c2c8cd3446abfeb8ba05aa36cc8c7b2e68ffd41424a"} Jul 10 19:07:54.667379 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:54.667332 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mm9bk" podStartSLOduration=3.902569707 podStartE2EDuration="25.667316853s" totalImagesPullingTime="21.764747146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:54.665833606 +0000 UTC m=+26.682291976" watchObservedRunningTime="2026-07-10 19:07:54.667316853 +0000 UTC m=+26.683775223" Jul 10 19:07:55.511413 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:55.511240 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:55.511413 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:55.511397 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:55.655815 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:55.655725 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:07:55.656651 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:55.656143 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerStarted","Data":"8246ee745dba5118fbf05e14241242bbe28211299a20a1e551c4f0713222f4c5"} Jul 10 19:07:55.656752 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:55.656737 2556 scope.go:117] "RemoveContainer" containerID="e9fac02fc201575f37d7b0a793bef2bdc0c5223e15ebcb35ff382ee27926d4c0" Jul 10 19:07:55.916791 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:55.916704 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:55.917588 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:55.917566 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:56.511852 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.511659 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:56.512053 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.511707 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:56.512053 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:56.511976 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:56.512053 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:56.512007 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:56.665615 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.665580 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:07:56.668417 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.668384 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" event={"ID":"14e7d49a-c59a-41c4-b0e7-357cc681a3a8","Type":"ContainerStarted","Data":"b45cdda101a6fd2e256389fd2c149a5af6730418d87917e712ff44b534f2e35c"} Jul 10 19:07:56.668555 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.668480 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:56.668636 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.668601 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:56.668687 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.668665 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:56.668791 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.668777 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:56.669563 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.669284 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fqzkj" Jul 10 19:07:56.692604 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.692457 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:56.693036 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.693006 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:07:56.707264 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:56.707164 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" podStartSLOduration=8.021676333 podStartE2EDuration="27.707143061s" totalImagesPullingTime="19.685466728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:07:56.700891755 +0000 UTC m=+28.717350151" watchObservedRunningTime="2026-07-10 19:07:56.707143061 +0000 UTC m=+28.723601433" Jul 10 19:07:57.511586 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:57.511557 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:57.511772 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:57.511680 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:57.719836 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:57.719792 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h42d9"] Jul 10 19:07:57.720276 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:57.719958 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:57.720276 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:57.720065 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:57.724463 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:57.724428 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7ng4r"] Jul 10 19:07:57.724613 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:57.724502 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cn5r4"] Jul 10 19:07:57.724613 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:57.724593 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:57.724742 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:57.724695 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:57.724791 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:57.724781 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:57.724898 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:57.724858 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:58.674168 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:58.673732 2556 generic.go:356] "Generic (PLEG): container finished" podID="668bc7ba-bf22-4bb2-ae0f-4c94bd912884" containerID="0f9805c8a56b219c4a84321cbe68e0f394044783c182106bec4bb2fb1e444ccd" exitCode=0 Jul 10 19:07:58.674168 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:58.674150 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" event={"ID":"668bc7ba-bf22-4bb2-ae0f-4c94bd912884","Type":"ContainerDied","Data":"0f9805c8a56b219c4a84321cbe68e0f394044783c182106bec4bb2fb1e444ccd"} Jul 10 19:07:59.511962 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:59.511933 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:07:59.511962 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:59.511966 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:07:59.512400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:59.511974 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:07:59.512400 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:59.512049 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:07:59.512400 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:59.512155 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:07:59.512400 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:07:59.512216 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:07:59.679237 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:59.679149 2556 generic.go:356] "Generic (PLEG): container finished" podID="668bc7ba-bf22-4bb2-ae0f-4c94bd912884" containerID="e7063f909836d5be886200b91032d24e3cd101a9fff47bb0a215f0007821ecf7" exitCode=0 Jul 10 19:07:59.679237 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:07:59.679228 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" event={"ID":"668bc7ba-bf22-4bb2-ae0f-4c94bd912884","Type":"ContainerDied","Data":"e7063f909836d5be886200b91032d24e3cd101a9fff47bb0a215f0007821ecf7"} Jul 10 19:08:00.684435 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:00.684181 2556 generic.go:356] "Generic (PLEG): container finished" podID="668bc7ba-bf22-4bb2-ae0f-4c94bd912884" containerID="b4ff87473d4c0a84efca0848ea4751da6a5dca4979dcdd9535f4b1dad6063b20" exitCode=0 Jul 10 19:08:00.684435 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:00.684257 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" event={"ID":"668bc7ba-bf22-4bb2-ae0f-4c94bd912884","Type":"ContainerDied","Data":"b4ff87473d4c0a84efca0848ea4751da6a5dca4979dcdd9535f4b1dad6063b20"} Jul 10 19:08:01.511165 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:01.511120 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:08:01.511165 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:01.511172 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:08:01.511425 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:01.511123 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:08:01.511425 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:01.511270 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42d9" podUID="9c4bb811-9e79-4b09-84fe-a6e7debe4cee" Jul 10 19:08:01.511425 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:01.511355 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cn5r4" podUID="bf3e3721-db7c-4586-ba94-72732ce64494" Jul 10 19:08:01.511567 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:01.511464 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ng4r" podUID="feb3ac88-148f-4921-9c03-dbe7f3d3da96" Jul 10 19:08:02.140864 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:02.140826 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:08:02.141414 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:02.140985 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 19:08:02.141414 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:02.141054 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret podName:9c4bb811-9e79-4b09-84fe-a6e7debe4cee nodeName:}" failed. No retries permitted until 2026-07-10 19:08:34.141038485 +0000 UTC m=+66.157496837 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret") pod "global-pull-secret-syncer-h42d9" (UID: "9c4bb811-9e79-4b09-84fe-a6e7debe4cee") : object "kube-system"/"original-pull-secret" not registered Jul 10 19:08:02.241709 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:02.241675 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:08:02.241912 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:02.241793 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:08:02.241912 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:02.241853 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs podName:feb3ac88-148f-4921-9c03-dbe7f3d3da96 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:34.241836086 +0000 UTC m=+66.258294437 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs") pod "network-metrics-daemon-7ng4r" (UID: "feb3ac88-148f-4921-9c03-dbe7f3d3da96") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 19:08:02.342018 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:02.341983 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:08:02.342211 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:02.342165 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 19:08:02.342211 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:02.342184 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 19:08:02.342211 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:02.342193 2556 projected.go:196] Error preparing data for projected volume kube-api-access-btjgd for pod openshift-network-diagnostics/network-check-target-cn5r4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:08:02.342353 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:02.342239 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd podName:bf3e3721-db7c-4586-ba94-72732ce64494 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:34.342226083 +0000 UTC m=+66.358684431 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-btjgd" (UniqueName: "kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd") pod "network-check-target-cn5r4" (UID: "bf3e3721-db7c-4586-ba94-72732ce64494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 19:08:03.256076 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.256046 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-73.ec2.internal" event="NodeReady" Jul 10 19:08:03.256508 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.256170 2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 19:08:03.289470 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.289437 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g"] Jul 10 19:08:03.294798 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.293984 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht"] Jul 10 19:08:03.294982 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.294803 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:03.297344 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.297322 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cdc884548-n9267"] Jul 10 19:08:03.297485 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.297457 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.297974 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.297953 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.298141 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.298116 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-f5smk\"" Jul 10 19:08:03.298250 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.298179 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 19:08:03.299455 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.299437 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 19:08:03.300069 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.300055 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.300239 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.300224 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn"] Jul 10 19:08:03.300334 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.300321 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.300643 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.300551 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 10 19:08:03.300804 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.300787 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.300861 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.300823 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-nv8mx\"" Jul 10 19:08:03.300924 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.300895 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.302778 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.302492 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 10 19:08:03.302778 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.302749 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lqfvm\"" Jul 10 19:08:03.302987 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.302756 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 19:08:03.303964 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.303410 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 19:08:03.303964 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.303478 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 19:08:03.306714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.306687 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-gxdsf"] Jul 10 19:08:03.306714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.306849 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:03.309219 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.309195 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.309723 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.309686 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 10 19:08:03.309914 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.309859 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nlsh4\"" Jul 10 19:08:03.309996 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.309947 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.310247 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.310226 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 19:08:03.310503 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.310480 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2"] Jul 10 19:08:03.310611 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.310597 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.313018 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.312986 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.313557 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.313218 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 10 19:08:03.314518 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.313985 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-w2lkm\"" Jul 10 19:08:03.314518 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.314335 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 10 19:08:03.316737 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.316711 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff"] Jul 10 19:08:03.317023 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.317003 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.317713 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.317692 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.318682 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.318659 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 10 19:08:03.320536 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.320261 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.321063 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.321039 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 19:08:03.321334 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.321261 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.321511 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.321492 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5c65fc6986-nkwgn"] Jul 10 19:08:03.321847 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.321826 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-d6h5t\"" Jul 10 19:08:03.321985 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.321967 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff" Jul 10 19:08:03.322099 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.321502 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 19:08:03.324928 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.324906 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.325105 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.324966 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.325105 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.325001 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-twfv7\"" Jul 10 19:08:03.325480 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.325457 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-gqxwz"] Jul 10 19:08:03.326147 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.326125 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.328576 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.328525 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.328576 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.328542 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 19:08:03.328828 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.328810 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bjqn6\"" Jul 10 19:08:03.330583 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.329090 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 19:08:03.330583 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.329210 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.330583 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.329541 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9"] Jul 10 19:08:03.330583 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.329588 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 19:08:03.330583 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.330082 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.330924 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.329544 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 19:08:03.332288 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.332269 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.332522 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.332482 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.333030 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.333009 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 19:08:03.333139 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.333072 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 19:08:03.333220 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.333200 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g"] Jul 10 19:08:03.333326 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.333237 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fqbbz"] Jul 10 19:08:03.333381 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.333368 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4q68t\"" Jul 10 19:08:03.333432 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.333368 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9" Jul 10 19:08:03.341854 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.341812 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.342042 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.341820 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.342684 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.342658 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cdtsd\"" Jul 10 19:08:03.345252 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.345226 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pgqt4"] Jul 10 19:08:03.345390 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.345295 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 19:08:03.345390 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.345356 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.347536 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.347517 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-z5rf9\"" Jul 10 19:08:03.347651 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.347554 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 19:08:03.347651 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.347522 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 19:08:03.348956 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.348938 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn"] Jul 10 19:08:03.349054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.348969 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2"] Jul 10 19:08:03.349054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.348986 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cdc884548-n9267"] Jul 10 19:08:03.349054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.349005 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-gxdsf"] Jul 10 19:08:03.349054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.349023 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-gqxwz"] Jul 10 19:08:03.349054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.349041 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fqbbz"] Jul 10 19:08:03.349281 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.349062 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht"] Jul 10 19:08:03.349281 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.349070 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pgqt4" Jul 10 19:08:03.349281 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.349086 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff"] Jul 10 19:08:03.349281 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.349106 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pgqt4"] Jul 10 19:08:03.349281 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.349121 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9"] Jul 10 19:08:03.349281 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.349138 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c65fc6986-nkwgn"] Jul 10 19:08:03.351228 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.351070 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 19:08:03.351680 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.351662 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4kdxh\"" Jul 10 19:08:03.351771 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.351686 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 19:08:03.351771 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.351760 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 19:08:03.449254 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449217 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-trusted-ca\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.449478 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449286 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j24hs\" (UniqueName: \"kubernetes.io/projected/200df66b-8fb3-408f-b765-520943132d83-kube-api-access-j24hs\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:03.449478 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449319 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88b8r\" (UniqueName: \"kubernetes.io/projected/65b055a9-5e50-44ee-9e0a-000735040140-kube-api-access-88b8r\") pod \"kube-storage-version-migrator-operator-799cb7f958-pzvv2\" (UID: \"65b055a9-5e50-44ee-9e0a-000735040140\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.449478 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449370 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7c5h\" (UniqueName: \"kubernetes.io/projected/9125148c-8264-4c42-bb5a-a811fc4f5588-kube-api-access-w7c5h\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.449478 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449392 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrj6j\" (UniqueName: \"kubernetes.io/projected/1045b744-a760-41ab-9e4d-ffe3ad4f80fb-kube-api-access-nrj6j\") pod \"ingress-canary-pgqt4\" (UID: \"1045b744-a760-41ab-9e4d-ffe3ad4f80fb\") " pod="openshift-ingress-canary/ingress-canary-pgqt4" Jul 10 19:08:03.449478 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449418 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzdz2\" (UniqueName: \"kubernetes.io/projected/2324f695-e128-4d57-950e-af3f5c363986-kube-api-access-qzdz2\") pod \"service-ca-operator-7695fc6c8c-7h4ht\" (UID: \"2324f695-e128-4d57-950e-af3f5c363986\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.449478 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449445 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dae262dc-c996-4f1d-b118-75619fbd3609-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:03.449478 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449469 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9125148c-8264-4c42-bb5a-a811fc4f5588-config-volume\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449488 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/324a3506-415f-46fc-94f0-47029379437d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449516 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-certificates\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449544 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbp25\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-kube-api-access-qbp25\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449643 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/324a3506-415f-46fc-94f0-47029379437d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449670 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449685 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/633078d5-6344-4d3c-b710-f205326c20f5-config\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449709 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-stats-auth\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449749 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9125148c-8264-4c42-bb5a-a811fc4f5588-tmp-dir\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449781 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-image-registry-private-configuration\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449806 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-bound-sa-token\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449826 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65b055a9-5e50-44ee-9e0a-000735040140-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-pzvv2\" (UID: \"65b055a9-5e50-44ee-9e0a-000735040140\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449843 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65b055a9-5e50-44ee-9e0a-000735040140-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-pzvv2\" (UID: \"65b055a9-5e50-44ee-9e0a-000735040140\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.449895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449865 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449910 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2324f695-e128-4d57-950e-af3f5c363986-config\") pod \"service-ca-operator-7695fc6c8c-7h4ht\" (UID: \"2324f695-e128-4d57-950e-af3f5c363986\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449937 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm765\" (UniqueName: \"kubernetes.io/projected/dae262dc-c996-4f1d-b118-75619fbd3609-kube-api-access-zm765\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449954 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.449977 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8rk\" (UniqueName: \"kubernetes.io/projected/633078d5-6344-4d3c-b710-f205326c20f5-kube-api-access-6p8rk\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450042 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-default-certificate\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450067 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450085 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vzqz\" (UniqueName: \"kubernetes.io/projected/1510cc81-e171-470a-bc51-ab3243a880b7-kube-api-access-7vzqz\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450107 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/324a3506-415f-46fc-94f0-47029379437d-snapshots\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450196 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a7e3559-1409-44cf-b46a-af8e91f48f75-ca-trust-extracted\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450240 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1045b744-a760-41ab-9e4d-ffe3ad4f80fb-cert\") pod \"ingress-canary-pgqt4\" (UID: \"1045b744-a760-41ab-9e4d-ffe3ad4f80fb\") " pod="openshift-ingress-canary/ingress-canary-pgqt4" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450314 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/633078d5-6344-4d3c-b710-f205326c20f5-trusted-ca\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450346 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbbz2\" (UniqueName: \"kubernetes.io/projected/230092b8-b633-4800-8567-7b7064570b72-kube-api-access-gbbz2\") pod \"volume-data-source-validator-78495597f6-n2fff\" (UID: \"230092b8-b633-4800-8567-7b7064570b72\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450375 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/324a3506-415f-46fc-94f0-47029379437d-tmp\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450391 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/324a3506-415f-46fc-94f0-47029379437d-serving-cert\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.450575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450406 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lwdn\" (UniqueName: \"kubernetes.io/projected/324a3506-415f-46fc-94f0-47029379437d-kube-api-access-2lwdn\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.451199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450423 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2324f695-e128-4d57-950e-af3f5c363986-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7h4ht\" (UID: \"2324f695-e128-4d57-950e-af3f5c363986\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.451199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450451 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.451199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450477 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/633078d5-6344-4d3c-b710-f205326c20f5-serving-cert\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.451199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450509 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:03.451199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450532 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57gln\" (UniqueName: \"kubernetes.io/projected/ea1781c2-6d8a-412b-b3bc-0fd8f1bf735d-kube-api-access-57gln\") pod \"network-check-source-869dcfcdcd-vbjz9\" (UID: \"ea1781c2-6d8a-412b-b3bc-0fd8f1bf735d\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9" Jul 10 19:08:03.451199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.450551 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-installation-pull-secrets\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.464582 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.464542 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fqzkj_da9b854d-8ef1-4986-841d-25e45fd65bab/konnectivity-agent/0.log" limit=1024 Jul 10 19:08:03.511427 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.511343 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:08:03.511612 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.511342 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:08:03.512218 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.511342 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:08:03.514775 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.514734 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-55ch6\"" Jul 10 19:08:03.514948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.514908 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xwfmf\"" Jul 10 19:08:03.515067 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.515050 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 19:08:03.515183 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.515160 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 19:08:03.551397 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551363 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzdz2\" (UniqueName: \"kubernetes.io/projected/2324f695-e128-4d57-950e-af3f5c363986-kube-api-access-qzdz2\") pod \"service-ca-operator-7695fc6c8c-7h4ht\" (UID: \"2324f695-e128-4d57-950e-af3f5c363986\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551423 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dae262dc-c996-4f1d-b118-75619fbd3609-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551448 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9125148c-8264-4c42-bb5a-a811fc4f5588-config-volume\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551465 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/324a3506-415f-46fc-94f0-47029379437d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551480 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-certificates\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551495 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbp25\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-kube-api-access-qbp25\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551537 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/324a3506-415f-46fc-94f0-47029379437d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551561 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551586 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/633078d5-6344-4d3c-b710-f205326c20f5-config\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551612 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-stats-auth\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.551640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551638 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9125148c-8264-4c42-bb5a-a811fc4f5588-tmp-dir\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551664 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-image-registry-private-configuration\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551686 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-bound-sa-token\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551716 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65b055a9-5e50-44ee-9e0a-000735040140-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-pzvv2\" (UID: \"65b055a9-5e50-44ee-9e0a-000735040140\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551740 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65b055a9-5e50-44ee-9e0a-000735040140-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-pzvv2\" (UID: \"65b055a9-5e50-44ee-9e0a-000735040140\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551768 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551789 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2324f695-e128-4d57-950e-af3f5c363986-config\") pod \"service-ca-operator-7695fc6c8c-7h4ht\" (UID: \"2324f695-e128-4d57-950e-af3f5c363986\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551819 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm765\" (UniqueName: \"kubernetes.io/projected/dae262dc-c996-4f1d-b118-75619fbd3609-kube-api-access-zm765\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551843 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551892 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8rk\" (UniqueName: \"kubernetes.io/projected/633078d5-6344-4d3c-b710-f205326c20f5-kube-api-access-6p8rk\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.551742 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551930 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-default-certificate\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.551948 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdc884548-n9267: secret "image-registry-tls" not found Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551959 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.551987 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vzqz\" (UniqueName: \"kubernetes.io/projected/1510cc81-e171-470a-bc51-ab3243a880b7-kube-api-access-7vzqz\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552010 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/324a3506-415f-46fc-94f0-47029379437d-snapshots\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552045 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a7e3559-1409-44cf-b46a-af8e91f48f75-ca-trust-extracted\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.552158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552065 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1045b744-a760-41ab-9e4d-ffe3ad4f80fb-cert\") pod \"ingress-canary-pgqt4\" (UID: \"1045b744-a760-41ab-9e4d-ffe3ad4f80fb\") " pod="openshift-ingress-canary/ingress-canary-pgqt4" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552088 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/633078d5-6344-4d3c-b710-f205326c20f5-trusted-ca\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552113 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbbz2\" (UniqueName: \"kubernetes.io/projected/230092b8-b633-4800-8567-7b7064570b72-kube-api-access-gbbz2\") pod \"volume-data-source-validator-78495597f6-n2fff\" (UID: \"230092b8-b633-4800-8567-7b7064570b72\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552138 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/324a3506-415f-46fc-94f0-47029379437d-tmp\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552157 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/324a3506-415f-46fc-94f0-47029379437d-serving-cert\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552184 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lwdn\" (UniqueName: \"kubernetes.io/projected/324a3506-415f-46fc-94f0-47029379437d-kube-api-access-2lwdn\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552208 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2324f695-e128-4d57-950e-af3f5c363986-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7h4ht\" (UID: \"2324f695-e128-4d57-950e-af3f5c363986\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552233 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552257 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/633078d5-6344-4d3c-b710-f205326c20f5-serving-cert\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552294 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552317 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57gln\" (UniqueName: \"kubernetes.io/projected/ea1781c2-6d8a-412b-b3bc-0fd8f1bf735d-kube-api-access-57gln\") pod \"network-check-source-869dcfcdcd-vbjz9\" (UID: \"ea1781c2-6d8a-412b-b3bc-0fd8f1bf735d\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552349 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-installation-pull-secrets\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552360 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dae262dc-c996-4f1d-b118-75619fbd3609-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552382 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-trusted-ca\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552386 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-certificates\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.552957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552409 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j24hs\" (UniqueName: \"kubernetes.io/projected/200df66b-8fb3-408f-b765-520943132d83-kube-api-access-j24hs\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552436 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88b8r\" (UniqueName: \"kubernetes.io/projected/65b055a9-5e50-44ee-9e0a-000735040140-kube-api-access-88b8r\") pod \"kube-storage-version-migrator-operator-799cb7f958-pzvv2\" (UID: \"65b055a9-5e50-44ee-9e0a-000735040140\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552460 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7c5h\" (UniqueName: \"kubernetes.io/projected/9125148c-8264-4c42-bb5a-a811fc4f5588-kube-api-access-w7c5h\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.552477 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls podName:1a7e3559-1409-44cf-b46a-af8e91f48f75 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:04.052455938 +0000 UTC m=+36.068914304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls") pod "image-registry-6cdc884548-n9267" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75") : secret "image-registry-tls" not found Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552534 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrj6j\" (UniqueName: \"kubernetes.io/projected/1045b744-a760-41ab-9e4d-ffe3ad4f80fb-kube-api-access-nrj6j\") pod \"ingress-canary-pgqt4\" (UID: \"1045b744-a760-41ab-9e4d-ffe3ad4f80fb\") " pod="openshift-ingress-canary/ingress-canary-pgqt4" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552677 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9125148c-8264-4c42-bb5a-a811fc4f5588-tmp-dir\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552724 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/324a3506-415f-46fc-94f0-47029379437d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.552808 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:04.052790743 +0000 UTC m=+36.069249109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : configmap references non-existent config key: service-ca.crt Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552830 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/633078d5-6344-4d3c-b710-f205326c20f5-config\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.553259 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/324a3506-415f-46fc-94f0-47029379437d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.553275 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/324a3506-415f-46fc-94f0-47029379437d-tmp\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.553358 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2324f695-e128-4d57-950e-af3f5c363986-config\") pod \"service-ca-operator-7695fc6c8c-7h4ht\" (UID: \"2324f695-e128-4d57-950e-af3f5c363986\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.553451 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.553500 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls podName:200df66b-8fb3-408f-b765-520943132d83 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:04.053483463 +0000 UTC m=+36.069941814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls") pod "cluster-samples-operator-65db858894-kqtfn" (UID: "200df66b-8fb3-408f-b765-520943132d83") : secret "samples-operator-tls" not found Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.553536 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65b055a9-5e50-44ee-9e0a-000735040140-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-pzvv2\" (UID: \"65b055a9-5e50-44ee-9e0a-000735040140\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.553599 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.553541 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:08:03.554237 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.553601 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls podName:9125148c-8264-4c42-bb5a-a811fc4f5588 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:04.053584659 +0000 UTC m=+36.070043028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls") pod "dns-default-fqbbz" (UID: "9125148c-8264-4c42-bb5a-a811fc4f5588") : secret "dns-default-metrics-tls" not found Jul 10 19:08:03.554237 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.553906 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a7e3559-1409-44cf-b46a-af8e91f48f75-ca-trust-extracted\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.555374 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/633078d5-6344-4d3c-b710-f205326c20f5-trusted-ca\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.555582 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.555650 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls podName:dae262dc-c996-4f1d-b118-75619fbd3609 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:04.055626019 +0000 UTC m=+36.072084384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bn47g" (UID: "dae262dc-c996-4f1d-b118-75619fbd3609") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.557601 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-trusted-ca\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.552157 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9125148c-8264-4c42-bb5a-a811fc4f5588-config-volume\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.558811 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/324a3506-415f-46fc-94f0-47029379437d-serving-cert\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.559056 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-stats-auth\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.559066 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1045b744-a760-41ab-9e4d-ffe3ad4f80fb-cert\") pod \"ingress-canary-pgqt4\" (UID: \"1045b744-a760-41ab-9e4d-ffe3ad4f80fb\") " pod="openshift-ingress-canary/ingress-canary-pgqt4" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.559197 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:03.559261 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:04.059243202 +0000 UTC m=+36.075701553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : secret "router-metrics-certs-default" not found Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.559357 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-installation-pull-secrets\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.559506 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/324a3506-415f-46fc-94f0-47029379437d-snapshots\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.559717 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-image-registry-private-configuration\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.560026 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/633078d5-6344-4d3c-b710-f205326c20f5-serving-cert\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.561233 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzdz2\" (UniqueName: \"kubernetes.io/projected/2324f695-e128-4d57-950e-af3f5c363986-kube-api-access-qzdz2\") pod \"service-ca-operator-7695fc6c8c-7h4ht\" (UID: \"2324f695-e128-4d57-950e-af3f5c363986\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.562270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.561601 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-default-certificate\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.563595 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.563561 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fqzkj_da9b854d-8ef1-4986-841d-25e45fd65bab/konnectivity-agent/0.log" limit=1024 Jul 10 19:08:03.563846 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.563824 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65b055a9-5e50-44ee-9e0a-000735040140-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-pzvv2\" (UID: \"65b055a9-5e50-44ee-9e0a-000735040140\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.564356 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.564329 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2324f695-e128-4d57-950e-af3f5c363986-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7h4ht\" (UID: \"2324f695-e128-4d57-950e-af3f5c363986\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.567675 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.567643 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8rk\" (UniqueName: \"kubernetes.io/projected/633078d5-6344-4d3c-b710-f205326c20f5-kube-api-access-6p8rk\") pod \"console-operator-7866497494-gxdsf\" (UID: \"633078d5-6344-4d3c-b710-f205326c20f5\") " pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.568401 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.568023 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j24hs\" (UniqueName: \"kubernetes.io/projected/200df66b-8fb3-408f-b765-520943132d83-kube-api-access-j24hs\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:03.568999 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.568672 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrj6j\" (UniqueName: \"kubernetes.io/projected/1045b744-a760-41ab-9e4d-ffe3ad4f80fb-kube-api-access-nrj6j\") pod \"ingress-canary-pgqt4\" (UID: \"1045b744-a760-41ab-9e4d-ffe3ad4f80fb\") " pod="openshift-ingress-canary/ingress-canary-pgqt4" Jul 10 19:08:03.568999 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.568946 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm765\" (UniqueName: \"kubernetes.io/projected/dae262dc-c996-4f1d-b118-75619fbd3609-kube-api-access-zm765\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:03.570115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.570060 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbp25\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-kube-api-access-qbp25\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.570983 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.570958 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-bound-sa-token\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:03.571089 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.571053 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbbz2\" (UniqueName: \"kubernetes.io/projected/230092b8-b633-4800-8567-7b7064570b72-kube-api-access-gbbz2\") pod \"volume-data-source-validator-78495597f6-n2fff\" (UID: \"230092b8-b633-4800-8567-7b7064570b72\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff" Jul 10 19:08:03.571999 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.571975 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vzqz\" (UniqueName: \"kubernetes.io/projected/1510cc81-e171-470a-bc51-ab3243a880b7-kube-api-access-7vzqz\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:03.572286 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.572262 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7c5h\" (UniqueName: \"kubernetes.io/projected/9125148c-8264-4c42-bb5a-a811fc4f5588-kube-api-access-w7c5h\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:03.572352 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.572331 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88b8r\" (UniqueName: \"kubernetes.io/projected/65b055a9-5e50-44ee-9e0a-000735040140-kube-api-access-88b8r\") pod \"kube-storage-version-migrator-operator-799cb7f958-pzvv2\" (UID: \"65b055a9-5e50-44ee-9e0a-000735040140\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.572497 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.572478 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57gln\" (UniqueName: \"kubernetes.io/projected/ea1781c2-6d8a-412b-b3bc-0fd8f1bf735d-kube-api-access-57gln\") pod \"network-check-source-869dcfcdcd-vbjz9\" (UID: \"ea1781c2-6d8a-412b-b3bc-0fd8f1bf735d\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9" Jul 10 19:08:03.573640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.573614 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lwdn\" (UniqueName: \"kubernetes.io/projected/324a3506-415f-46fc-94f0-47029379437d-kube-api-access-2lwdn\") pod \"insights-operator-68d9dd94cf-gqxwz\" (UID: \"324a3506-415f-46fc-94f0-47029379437d\") " pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.619683 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.619645 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" Jul 10 19:08:03.653854 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.653822 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:03.661404 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.661373 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" Jul 10 19:08:03.666532 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.666512 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff" Jul 10 19:08:03.678850 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.678818 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" Jul 10 19:08:03.688444 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.688028 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9" Jul 10 19:08:03.699033 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.698815 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pgqt4" Jul 10 19:08:03.846593 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.845443 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht"] Jul 10 19:08:03.913619 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.912679 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-gxdsf"] Jul 10 19:08:03.943166 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.943078 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-gqxwz"] Jul 10 19:08:03.948334 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:03.948304 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod324a3506_415f_46fc_94f0_47029379437d.slice/crio-0cc606639dc3b5c9d37d884687da86e8eb8c0228de431af10e66165adae9cfe3 WatchSource:0}: Error finding container 0cc606639dc3b5c9d37d884687da86e8eb8c0228de431af10e66165adae9cfe3: Status 404 returned error can't find the container with id 0cc606639dc3b5c9d37d884687da86e8eb8c0228de431af10e66165adae9cfe3 Jul 10 19:08:03.963497 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.963326 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2"] Jul 10 19:08:03.966520 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.966491 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff"] Jul 10 19:08:03.972804 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:03.972776 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230092b8_b633_4800_8567_7b7064570b72.slice/crio-b80c8b9fb8bc78b7975ce6c84c8555c85b24915395e2c1b7dab8fca1cee7052d WatchSource:0}: Error finding container b80c8b9fb8bc78b7975ce6c84c8555c85b24915395e2c1b7dab8fca1cee7052d: Status 404 returned error can't find the container with id b80c8b9fb8bc78b7975ce6c84c8555c85b24915395e2c1b7dab8fca1cee7052d Jul 10 19:08:03.977906 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.977450 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9"] Jul 10 19:08:03.980755 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:03.980730 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1781c2_6d8a_412b_b3bc_0fd8f1bf735d.slice/crio-6e0040570da270bcc06500f80cfe24d2c97ac21c4efe4eb250d9ae18e0c94d67 WatchSource:0}: Error finding container 6e0040570da270bcc06500f80cfe24d2c97ac21c4efe4eb250d9ae18e0c94d67: Status 404 returned error can't find the container with id 6e0040570da270bcc06500f80cfe24d2c97ac21c4efe4eb250d9ae18e0c94d67 Jul 10 19:08:03.991548 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:03.991519 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pgqt4"] Jul 10 19:08:03.995027 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:03.994968 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1045b744_a760_41ab_9e4d_ffe3ad4f80fb.slice/crio-c4934625ac471fd5177dda8bf2777867030712478cb44adb9f35b56844efb813 WatchSource:0}: Error finding container c4934625ac471fd5177dda8bf2777867030712478cb44adb9f35b56844efb813: Status 404 returned error can't find the container with id c4934625ac471fd5177dda8bf2777867030712478cb44adb9f35b56844efb813 Jul 10 19:08:04.065090 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.065053 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:04.065090 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.065094 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:04.065320 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.065118 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:04.065320 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065217 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:04.065320 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065222 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:08:04.065320 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.065250 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:04.065320 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065265 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls podName:dae262dc-c996-4f1d-b118-75619fbd3609 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:05.065251466 +0000 UTC m=+37.081709818 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bn47g" (UID: "dae262dc-c996-4f1d-b118-75619fbd3609") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:04.065320 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065281 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls podName:9125148c-8264-4c42-bb5a-a811fc4f5588 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:05.065275217 +0000 UTC m=+37.081733565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls") pod "dns-default-fqbbz" (UID: "9125148c-8264-4c42-bb5a-a811fc4f5588") : secret "dns-default-metrics-tls" not found Jul 10 19:08:04.065320 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065311 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:05.065295223 +0000 UTC m=+37.081753572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : configmap references non-existent config key: service-ca.crt Jul 10 19:08:04.065661 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065330 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:08:04.065661 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.065368 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:04.065661 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065385 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:05.065369882 +0000 UTC m=+37.081828237 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : secret "router-metrics-certs-default" not found Jul 10 19:08:04.065661 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065431 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 19:08:04.065661 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.065487 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:04.065661 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065533 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls podName:200df66b-8fb3-408f-b765-520943132d83 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:05.065526537 +0000 UTC m=+37.081984886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls") pod "cluster-samples-operator-65db858894-kqtfn" (UID: "200df66b-8fb3-408f-b765-520943132d83") : secret "samples-operator-tls" not found Jul 10 19:08:04.065661 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065587 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:08:04.065661 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065600 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdc884548-n9267: secret "image-registry-tls" not found Jul 10 19:08:04.065661 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:04.065645 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls podName:1a7e3559-1409-44cf-b46a-af8e91f48f75 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:05.065633411 +0000 UTC m=+37.082091777 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls") pod "image-registry-6cdc884548-n9267" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75") : secret "image-registry-tls" not found Jul 10 19:08:04.697930 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.697890 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pgqt4" event={"ID":"1045b744-a760-41ab-9e4d-ffe3ad4f80fb","Type":"ContainerStarted","Data":"c4934625ac471fd5177dda8bf2777867030712478cb44adb9f35b56844efb813"} Jul 10 19:08:04.699345 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.699312 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9" event={"ID":"ea1781c2-6d8a-412b-b3bc-0fd8f1bf735d","Type":"ContainerStarted","Data":"6e0040570da270bcc06500f80cfe24d2c97ac21c4efe4eb250d9ae18e0c94d67"} Jul 10 19:08:04.700605 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.700573 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" event={"ID":"65b055a9-5e50-44ee-9e0a-000735040140","Type":"ContainerStarted","Data":"79eba108eff8e327bd9ccb421f59ea5562eba1428fe872f742d777e7b55e02c6"} Jul 10 19:08:04.701923 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.701891 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" event={"ID":"2324f695-e128-4d57-950e-af3f5c363986","Type":"ContainerStarted","Data":"d8820b69d06f7c84d535476a280ade844e70169a3e779ade5d988687479715f0"} Jul 10 19:08:04.703208 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.703182 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" event={"ID":"324a3506-415f-46fc-94f0-47029379437d","Type":"ContainerStarted","Data":"0cc606639dc3b5c9d37d884687da86e8eb8c0228de431af10e66165adae9cfe3"} Jul 10 19:08:04.704355 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.704328 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gxdsf" event={"ID":"633078d5-6344-4d3c-b710-f205326c20f5","Type":"ContainerStarted","Data":"7aa7f57846bc26f06eb41ab7a8f775de04d677d32e3f960e392f275348515bbf"} Jul 10 19:08:04.705457 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:04.705435 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff" event={"ID":"230092b8-b633-4800-8567-7b7064570b72","Type":"ContainerStarted","Data":"b80c8b9fb8bc78b7975ce6c84c8555c85b24915395e2c1b7dab8fca1cee7052d"} Jul 10 19:08:05.073440 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:05.073402 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:05.073440 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:05.073445 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:05.073688 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:05.073484 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:05.073688 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:05.073529 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:05.073688 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:05.073575 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:05.073688 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073605 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:07.073586928 +0000 UTC m=+39.090045290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : configmap references non-existent config key: service-ca.crt Jul 10 19:08:05.073688 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073639 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:08:05.073688 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073648 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:05.073688 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073691 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:08:05.073688 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073643 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 19:08:05.074088 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073700 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls podName:9125148c-8264-4c42-bb5a-a811fc4f5588 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:07.073681516 +0000 UTC m=+39.090139881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls") pod "dns-default-fqbbz" (UID: "9125148c-8264-4c42-bb5a-a811fc4f5588") : secret "dns-default-metrics-tls" not found Jul 10 19:08:05.074088 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073743 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls podName:200df66b-8fb3-408f-b765-520943132d83 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:07.073720196 +0000 UTC m=+39.090178550 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls") pod "cluster-samples-operator-65db858894-kqtfn" (UID: "200df66b-8fb3-408f-b765-520943132d83") : secret "samples-operator-tls" not found Jul 10 19:08:05.074088 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:05.073727 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:05.074088 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073762 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls podName:dae262dc-c996-4f1d-b118-75619fbd3609 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:07.073751063 +0000 UTC m=+39.090209415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bn47g" (UID: "dae262dc-c996-4f1d-b118-75619fbd3609") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:05.074088 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073763 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:08:05.074088 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073776 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdc884548-n9267: secret "image-registry-tls" not found Jul 10 19:08:05.074088 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073783 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:07.073774585 +0000 UTC m=+39.090232943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : secret "router-metrics-certs-default" not found Jul 10 19:08:05.074088 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:05.073807 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls podName:1a7e3559-1409-44cf-b46a-af8e91f48f75 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:07.073797062 +0000 UTC m=+39.090255411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls") pod "image-registry-6cdc884548-n9267" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75") : secret "image-registry-tls" not found Jul 10 19:08:06.717492 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:06.717450 2556 generic.go:356] "Generic (PLEG): container finished" podID="668bc7ba-bf22-4bb2-ae0f-4c94bd912884" containerID="def7f0fdb20144d9fbcd5b53876c2c938496d1c788fccfb92788b7493f58a115" exitCode=0 Jul 10 19:08:06.717492 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:06.717499 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" event={"ID":"668bc7ba-bf22-4bb2-ae0f-4c94bd912884","Type":"ContainerDied","Data":"def7f0fdb20144d9fbcd5b53876c2c938496d1c788fccfb92788b7493f58a115"} Jul 10 19:08:07.093208 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:07.093120 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:07.093378 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:07.093228 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:07.093378 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:07.093263 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:07.093378 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:07.093288 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:07.093378 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093311 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 19:08:07.093378 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:07.093323 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:07.093378 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:07.093374 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:07.093758 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093394 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls podName:200df66b-8fb3-408f-b765-520943132d83 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:11.093372105 +0000 UTC m=+43.109830475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls") pod "cluster-samples-operator-65db858894-kqtfn" (UID: "200df66b-8fb3-408f-b765-520943132d83") : secret "samples-operator-tls" not found Jul 10 19:08:07.093758 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093491 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:08:07.093758 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093546 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:11.093530189 +0000 UTC m=+43.109988565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : secret "router-metrics-certs-default" not found Jul 10 19:08:07.093758 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093612 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:08:07.093758 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093624 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdc884548-n9267: secret "image-registry-tls" not found Jul 10 19:08:07.093758 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093659 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls podName:1a7e3559-1409-44cf-b46a-af8e91f48f75 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:11.093647955 +0000 UTC m=+43.110106343 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls") pod "image-registry-6cdc884548-n9267" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75") : secret "image-registry-tls" not found Jul 10 19:08:07.093758 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093755 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:11.093744454 +0000 UTC m=+43.110202822 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : configmap references non-existent config key: service-ca.crt Jul 10 19:08:07.094194 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093822 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:08:07.094194 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093831 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:07.094194 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093886 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls podName:9125148c-8264-4c42-bb5a-a811fc4f5588 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:11.093853135 +0000 UTC m=+43.110311489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls") pod "dns-default-fqbbz" (UID: "9125148c-8264-4c42-bb5a-a811fc4f5588") : secret "dns-default-metrics-tls" not found Jul 10 19:08:07.094194 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:07.093908 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls podName:dae262dc-c996-4f1d-b118-75619fbd3609 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:11.09389594 +0000 UTC m=+43.110354295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bn47g" (UID: "dae262dc-c996-4f1d-b118-75619fbd3609") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:07.728488 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:07.727432 2556 generic.go:356] "Generic (PLEG): container finished" podID="668bc7ba-bf22-4bb2-ae0f-4c94bd912884" containerID="d96f663e51cbd9426a50a7007038eb78dc0a347452c264d353b8514de132330f" exitCode=0 Jul 10 19:08:07.728488 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:07.727527 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" event={"ID":"668bc7ba-bf22-4bb2-ae0f-4c94bd912884","Type":"ContainerDied","Data":"d96f663e51cbd9426a50a7007038eb78dc0a347452c264d353b8514de132330f"} Jul 10 19:08:11.136045 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:11.135782 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:11.136075 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:11.136101 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.135990 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136129 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdc884548-n9267: secret "image-registry-tls" not found Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:11.136143 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136200 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls podName:1a7e3559-1409-44cf-b46a-af8e91f48f75 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:19.136179439 +0000 UTC m=+51.152637809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls") pod "image-registry-6cdc884548-n9267" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75") : secret "image-registry-tls" not found Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:11.136234 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136247 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136254 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136271 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:19.136261314 +0000 UTC m=+51.152719665 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : configmap references non-existent config key: service-ca.crt Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136298 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls podName:dae262dc-c996-4f1d-b118-75619fbd3609 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:19.136288502 +0000 UTC m=+51.152746853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bn47g" (UID: "dae262dc-c996-4f1d-b118-75619fbd3609") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:11.136315 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136328 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls podName:9125148c-8264-4c42-bb5a-a811fc4f5588 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:19.136307766 +0000 UTC m=+51.152766118 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls") pod "dns-default-fqbbz" (UID: "9125148c-8264-4c42-bb5a-a811fc4f5588") : secret "dns-default-metrics-tls" not found Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136371 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136370 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:08:11.136507 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136404 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls podName:200df66b-8fb3-408f-b765-520943132d83 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:19.136394198 +0000 UTC m=+51.152852550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls") pod "cluster-samples-operator-65db858894-kqtfn" (UID: "200df66b-8fb3-408f-b765-520943132d83") : secret "samples-operator-tls" not found Jul 10 19:08:11.137146 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:11.136439 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:19.13642222 +0000 UTC m=+51.152880571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : secret "router-metrics-certs-default" not found Jul 10 19:08:13.754533 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.754082 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff" event={"ID":"230092b8-b633-4800-8567-7b7064570b72","Type":"ContainerStarted","Data":"e3a9bffb9eb3ab054b9e468064013aef8770bd7fd291133ee566610a94d2215d"} Jul 10 19:08:13.757190 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.757111 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pgqt4" event={"ID":"1045b744-a760-41ab-9e4d-ffe3ad4f80fb","Type":"ContainerStarted","Data":"690fea043e23e2e7176827788685ff75e17c27576439b72ac660462ea1c0c7bf"} Jul 10 19:08:13.758758 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.758733 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9" event={"ID":"ea1781c2-6d8a-412b-b3bc-0fd8f1bf735d","Type":"ContainerStarted","Data":"5d5cc51e6c4a14ab4348f9a94fe3232273c867652dfc9ac372f01025687dae9e"} Jul 10 19:08:13.760430 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.760407 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" event={"ID":"65b055a9-5e50-44ee-9e0a-000735040140","Type":"ContainerStarted","Data":"1e5361aa86bb58c5637b026a4d4556d5e87389556f06627e1a8ff11f9c7c0122"} Jul 10 19:08:13.763346 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.762939 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" event={"ID":"2324f695-e128-4d57-950e-af3f5c363986","Type":"ContainerStarted","Data":"260563b01365c8cfcb51fca9702bc3f53ea71c237309fa57f6153aa1a76acf1e"} Jul 10 19:08:13.767306 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.767286 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" event={"ID":"668bc7ba-bf22-4bb2-ae0f-4c94bd912884","Type":"ContainerStarted","Data":"13c19a1c73157c1e4b23e7bf487053036945bb3dc5928fb81d3ed198d83532fe"} Jul 10 19:08:13.772084 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.770736 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-n2fff" podStartSLOduration=8.570008893 podStartE2EDuration="17.770719509s" totalImagesPullingTime="9.200710616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:13.76919478 +0000 UTC m=+45.785653153" watchObservedRunningTime="2026-07-10 19:08:13.770719509 +0000 UTC m=+45.787177882" Jul 10 19:08:13.772084 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.770788 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" event={"ID":"324a3506-415f-46fc-94f0-47029379437d","Type":"ContainerStarted","Data":"176e809e96c6d92ab990ac4bb91bc56c1d4c300c38c6aef49ef364d88aa1728d"} Jul 10 19:08:13.774953 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.774359 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/0.log" Jul 10 19:08:13.774953 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.774396 2556 generic.go:356] "Generic (PLEG): container finished" podID="633078d5-6344-4d3c-b710-f205326c20f5" containerID="768b419b5ba3c60dcbd1e6599567c47585e1714e4a18c18891435cb80a157630" exitCode=255 Jul 10 19:08:13.774953 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.774448 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gxdsf" event={"ID":"633078d5-6344-4d3c-b710-f205326c20f5","Type":"ContainerDied","Data":"768b419b5ba3c60dcbd1e6599567c47585e1714e4a18c18891435cb80a157630"} Jul 10 19:08:13.774953 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.774730 2556 scope.go:117] "RemoveContainer" containerID="768b419b5ba3c60dcbd1e6599567c47585e1714e4a18c18891435cb80a157630" Jul 10 19:08:13.819561 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.819100 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vbjz9" podStartSLOduration=6.190174075 podStartE2EDuration="15.819075964s" totalImagesPullingTime="9.628901889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:13.800505365 +0000 UTC m=+45.816963737" watchObservedRunningTime="2026-07-10 19:08:13.819075964 +0000 UTC m=+45.835534336" Jul 10 19:08:13.820226 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.820178 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" podStartSLOduration=8.397852257 podStartE2EDuration="17.820162493s" totalImagesPullingTime="9.422310236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:13.818554158 +0000 UTC m=+45.835012530" watchObservedRunningTime="2026-07-10 19:08:13.820162493 +0000 UTC m=+45.836620865" Jul 10 19:08:13.833453 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.833392 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pgqt4" podStartSLOduration=1.527886979 podStartE2EDuration="10.833370413s" totalImagesPullingTime="9.305483434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:13.832735179 +0000 UTC m=+45.849193552" watchObservedRunningTime="2026-07-10 19:08:13.833370413 +0000 UTC m=+45.849828777" Jul 10 19:08:13.856087 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.855721 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k4fbp" podStartSLOduration=13.156912611 podStartE2EDuration="44.855699212s" totalImagesPullingTime="31.698786601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:13.853978832 +0000 UTC m=+45.870437229" watchObservedRunningTime="2026-07-10 19:08:13.855699212 +0000 UTC m=+45.872157584" Jul 10 19:08:13.875164 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:13.874693 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" podStartSLOduration=8.548524862 podStartE2EDuration="17.874672841s" totalImagesPullingTime="9.326147979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:13.873184965 +0000 UTC m=+45.889643363" watchObservedRunningTime="2026-07-10 19:08:13.874672841 +0000 UTC m=+45.891131214" Jul 10 19:08:14.780280 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:14.780245 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:08:14.780725 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:14.780649 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/0.log" Jul 10 19:08:14.780725 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:14.780689 2556 generic.go:356] "Generic (PLEG): container finished" podID="633078d5-6344-4d3c-b710-f205326c20f5" containerID="f816faacf32a1d97e1a3c88cf4593b96fd89b66c194b055c26d459e3e37cf7b2" exitCode=255 Jul 10 19:08:14.780845 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:14.780799 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gxdsf" event={"ID":"633078d5-6344-4d3c-b710-f205326c20f5","Type":"ContainerDied","Data":"f816faacf32a1d97e1a3c88cf4593b96fd89b66c194b055c26d459e3e37cf7b2"} Jul 10 19:08:14.780922 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:14.780906 2556 scope.go:117] "RemoveContainer" containerID="768b419b5ba3c60dcbd1e6599567c47585e1714e4a18c18891435cb80a157630" Jul 10 19:08:14.781624 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:14.781172 2556 scope.go:117] "RemoveContainer" containerID="f816faacf32a1d97e1a3c88cf4593b96fd89b66c194b055c26d459e3e37cf7b2" Jul 10 19:08:14.781624 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:14.781371 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-gxdsf_openshift-console-operator(633078d5-6344-4d3c-b710-f205326c20f5)\"" pod="openshift-console-operator/console-operator-7866497494-gxdsf" podUID="633078d5-6344-4d3c-b710-f205326c20f5" Jul 10 19:08:14.798577 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:14.798526 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" podStartSLOduration=9.449754905 podStartE2EDuration="18.79850947s" totalImagesPullingTime="9.348754565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:13.912054115 +0000 UTC m=+45.928512488" watchObservedRunningTime="2026-07-10 19:08:14.79850947 +0000 UTC m=+46.814967864" Jul 10 19:08:15.099370 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.099275 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc"] Jul 10 19:08:15.132805 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.132773 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc"] Jul 10 19:08:15.132984 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.132952 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc" Jul 10 19:08:15.135848 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.135820 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-74sdm\"" Jul 10 19:08:15.136078 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.135970 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 10 19:08:15.136173 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.136155 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 10 19:08:15.176948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.176908 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw5tf\" (UniqueName: \"kubernetes.io/projected/3614c216-09cd-4b68-bdd9-42286a6c2454-kube-api-access-bw5tf\") pod \"migrator-7f59b54f6c-jhpkc\" (UID: \"3614c216-09cd-4b68-bdd9-42286a6c2454\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc" Jul 10 19:08:15.278219 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.278172 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw5tf\" (UniqueName: \"kubernetes.io/projected/3614c216-09cd-4b68-bdd9-42286a6c2454-kube-api-access-bw5tf\") pod \"migrator-7f59b54f6c-jhpkc\" (UID: \"3614c216-09cd-4b68-bdd9-42286a6c2454\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc" Jul 10 19:08:15.289049 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.289009 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw5tf\" (UniqueName: \"kubernetes.io/projected/3614c216-09cd-4b68-bdd9-42286a6c2454-kube-api-access-bw5tf\") pod \"migrator-7f59b54f6c-jhpkc\" (UID: \"3614c216-09cd-4b68-bdd9-42286a6c2454\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc" Jul 10 19:08:15.444453 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.444361 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc" Jul 10 19:08:15.476549 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.476493 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-l7g8z"] Jul 10 19:08:15.500366 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.500330 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l7g8z"] Jul 10 19:08:15.500687 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.500456 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.503013 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.502986 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vnt9v\"" Jul 10 19:08:15.503144 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.503021 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 19:08:15.503144 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.503021 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 19:08:15.581115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.581016 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c36cf591-5252-4695-8047-20f5a08d5908-data-volume\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.581115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.581091 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c36cf591-5252-4695-8047-20f5a08d5908-crio-socket\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.581115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.581119 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d97fr\" (UniqueName: \"kubernetes.io/projected/c36cf591-5252-4695-8047-20f5a08d5908-kube-api-access-d97fr\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.581385 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.581169 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.581385 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.581237 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.618281 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.618249 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc"] Jul 10 19:08:15.621179 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:15.621149 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3614c216_09cd_4b68_bdd9_42286a6c2454.slice/crio-e12853773d28feb4c56616caff99c8343631ff8a421af59bd2eb87f2b6f59a6f WatchSource:0}: Error finding container e12853773d28feb4c56616caff99c8343631ff8a421af59bd2eb87f2b6f59a6f: Status 404 returned error can't find the container with id e12853773d28feb4c56616caff99c8343631ff8a421af59bd2eb87f2b6f59a6f Jul 10 19:08:15.682456 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.682419 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c36cf591-5252-4695-8047-20f5a08d5908-data-volume\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.682654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.682474 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c36cf591-5252-4695-8047-20f5a08d5908-crio-socket\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.682654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.682506 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d97fr\" (UniqueName: \"kubernetes.io/projected/c36cf591-5252-4695-8047-20f5a08d5908-kube-api-access-d97fr\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.682654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.682552 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.682791 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:15.682674 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:08:15.682791 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.682678 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c36cf591-5252-4695-8047-20f5a08d5908-crio-socket\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.682791 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.682702 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.682791 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:15.682737 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls podName:c36cf591-5252-4695-8047-20f5a08d5908 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:16.182718217 +0000 UTC m=+48.199176589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7g8z" (UID: "c36cf591-5252-4695-8047-20f5a08d5908") : secret "insights-runtime-extractor-tls" not found Jul 10 19:08:15.682791 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:15.682756 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 19:08:15.682980 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:15.682820 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm podName:c36cf591-5252-4695-8047-20f5a08d5908 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:16.182803731 +0000 UTC m=+48.199262081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l7g8z" (UID: "c36cf591-5252-4695-8047-20f5a08d5908") : configmap "kube-rbac-proxy" not found Jul 10 19:08:15.682980 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.682835 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c36cf591-5252-4695-8047-20f5a08d5908-data-volume\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.691160 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.691133 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d97fr\" (UniqueName: \"kubernetes.io/projected/c36cf591-5252-4695-8047-20f5a08d5908-kube-api-access-d97fr\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:15.785181 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.785142 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc" event={"ID":"3614c216-09cd-4b68-bdd9-42286a6c2454","Type":"ContainerStarted","Data":"e12853773d28feb4c56616caff99c8343631ff8a421af59bd2eb87f2b6f59a6f"} Jul 10 19:08:15.786579 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.786557 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:08:15.786916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:15.786901 2556 scope.go:117] "RemoveContainer" containerID="f816faacf32a1d97e1a3c88cf4593b96fd89b66c194b055c26d459e3e37cf7b2" Jul 10 19:08:15.787080 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:15.787062 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-gxdsf_openshift-console-operator(633078d5-6344-4d3c-b710-f205326c20f5)\"" pod="openshift-console-operator/console-operator-7866497494-gxdsf" podUID="633078d5-6344-4d3c-b710-f205326c20f5" Jul 10 19:08:16.187746 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:16.187651 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:16.187914 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:16.187807 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 19:08:16.187914 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:16.187818 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:16.187914 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:16.187898 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm podName:c36cf591-5252-4695-8047-20f5a08d5908 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:17.187853558 +0000 UTC m=+49.204311939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l7g8z" (UID: "c36cf591-5252-4695-8047-20f5a08d5908") : configmap "kube-rbac-proxy" not found Jul 10 19:08:16.187914 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:16.187909 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:08:16.188072 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:16.187950 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls podName:c36cf591-5252-4695-8047-20f5a08d5908 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:17.187939094 +0000 UTC m=+49.204397447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7g8z" (UID: "c36cf591-5252-4695-8047-20f5a08d5908") : secret "insights-runtime-extractor-tls" not found Jul 10 19:08:17.197692 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:17.197645 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:17.197692 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:17.197700 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:17.198150 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:17.197779 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 19:08:17.198150 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:17.197817 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:08:17.198150 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:17.197845 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm podName:c36cf591-5252-4695-8047-20f5a08d5908 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:19.19783034 +0000 UTC m=+51.214288689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l7g8z" (UID: "c36cf591-5252-4695-8047-20f5a08d5908") : configmap "kube-rbac-proxy" not found Jul 10 19:08:17.198150 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:17.197899 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls podName:c36cf591-5252-4695-8047-20f5a08d5908 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:19.197860928 +0000 UTC m=+51.214319277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7g8z" (UID: "c36cf591-5252-4695-8047-20f5a08d5908") : secret "insights-runtime-extractor-tls" not found Jul 10 19:08:17.271136 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:17.271109 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ctm9t_669e056e-e810-451c-a00f-4e1c0a9d4c26/dns-node-resolver/0.log" Jul 10 19:08:17.795997 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:17.795956 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc" event={"ID":"3614c216-09cd-4b68-bdd9-42286a6c2454","Type":"ContainerStarted","Data":"4c3fd2a439fbdd8b79d35712be8f084fac2f535ed0d06292bb4c9c0c49e01ebe"} Jul 10 19:08:17.796168 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:17.796007 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc" event={"ID":"3614c216-09cd-4b68-bdd9-42286a6c2454","Type":"ContainerStarted","Data":"0dee244ca64b8f6eec60fb88c42b0507503194bc3e2401152d818adefcd34c95"} Jul 10 19:08:17.811289 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:17.811237 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-jhpkc" podStartSLOduration=1.05845332 podStartE2EDuration="2.811221522s" totalImagesPullingTime="1.752768202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:08:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:17.811141098 +0000 UTC m=+49.827599471" watchObservedRunningTime="2026-07-10 19:08:17.811221522 +0000 UTC m=+49.827679892" Jul 10 19:08:18.472670 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:18.472638 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p67tg_46f8b6c0-73ab-4684-91c6-04e358626759/node-ca/0.log" Jul 10 19:08:19.076587 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.076551 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pgqt4_1045b744-a760-41ab-9e4d-ffe3ad4f80fb/serve-healthcheck-canary/0.log" Jul 10 19:08:19.214824 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.214774 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:19.215022 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.214837 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:19.215022 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.214899 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:19.215022 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.214928 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:19.215022 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.214950 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:19.215022 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.214986 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:19.215022 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215008 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 19:08:19.215233 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215014 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 19:08:19.215233 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215012 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls podName:dae262dc-c996-4f1d-b118-75619fbd3609 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:35.21499498 +0000 UTC m=+67.231453329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-bn47g" (UID: "dae262dc-c996-4f1d-b118-75619fbd3609") : secret "cluster-monitoring-operator-tls" not found Jul 10 19:08:19.215233 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215080 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:08:19.215233 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215102 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:35.215083186 +0000 UTC m=+67.231541536 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : secret "router-metrics-certs-default" not found Jul 10 19:08:19.215233 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215123 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls podName:200df66b-8fb3-408f-b765-520943132d83 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:35.215111679 +0000 UTC m=+67.231570027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls") pod "cluster-samples-operator-65db858894-kqtfn" (UID: "200df66b-8fb3-408f-b765-520943132d83") : secret "samples-operator-tls" not found Jul 10 19:08:19.215233 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215143 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls podName:c36cf591-5252-4695-8047-20f5a08d5908 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:23.215133065 +0000 UTC m=+55.231591439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7g8z" (UID: "c36cf591-5252-4695-8047-20f5a08d5908") : secret "insights-runtime-extractor-tls" not found Jul 10 19:08:19.215233 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.215184 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:19.215233 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.215228 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:19.215509 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.215254 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:19.215509 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215258 2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 19:08:19.215509 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215273 2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cdc884548-n9267: secret "image-registry-tls" not found Jul 10 19:08:19.215509 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215311 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls podName:1a7e3559-1409-44cf-b46a-af8e91f48f75 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:35.215296889 +0000 UTC m=+67.231755238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls") pod "image-registry-6cdc884548-n9267" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75") : secret "image-registry-tls" not found Jul 10 19:08:19.215509 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215340 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 19:08:19.215509 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215354 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle podName:1510cc81-e171-470a-bc51-ab3243a880b7 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:35.215341284 +0000 UTC m=+67.231799633 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle") pod "router-default-5c65fc6986-nkwgn" (UID: "1510cc81-e171-470a-bc51-ab3243a880b7") : configmap references non-existent config key: service-ca.crt Jul 10 19:08:19.215509 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:19.215371 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls podName:9125148c-8264-4c42-bb5a-a811fc4f5588 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:35.215362003 +0000 UTC m=+67.231820379 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls") pod "dns-default-fqbbz" (UID: "9125148c-8264-4c42-bb5a-a811fc4f5588") : secret "dns-default-metrics-tls" not found Jul 10 19:08:19.215715 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.215532 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c36cf591-5252-4695-8047-20f5a08d5908-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:19.272948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.272918 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-jhpkc_3614c216-09cd-4b68-bdd9-42286a6c2454/migrator/0.log" Jul 10 19:08:19.473588 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.473506 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-jhpkc_3614c216-09cd-4b68-bdd9-42286a6c2454/graceful-termination/0.log" Jul 10 19:08:19.673979 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:19.673921 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-pzvv2_65b055a9-5e50-44ee-9e0a-000735040140/kube-storage-version-migrator-operator/0.log" Jul 10 19:08:23.253511 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:23.253471 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:23.254029 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:23.253655 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 19:08:23.254029 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:23.253743 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls podName:c36cf591-5252-4695-8047-20f5a08d5908 nodeName:}" failed. No retries permitted until 2026-07-10 19:08:31.253720906 +0000 UTC m=+63.270179270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7g8z" (UID: "c36cf591-5252-4695-8047-20f5a08d5908") : secret "insights-runtime-extractor-tls" not found Jul 10 19:08:23.654803 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:23.654711 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:23.654803 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:23.654805 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:23.655190 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:23.655177 2556 scope.go:117] "RemoveContainer" containerID="f816faacf32a1d97e1a3c88cf4593b96fd89b66c194b055c26d459e3e37cf7b2" Jul 10 19:08:23.655348 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:08:23.655329 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-gxdsf_openshift-console-operator(633078d5-6344-4d3c-b710-f205326c20f5)\"" pod="openshift-console-operator/console-operator-7866497494-gxdsf" podUID="633078d5-6344-4d3c-b710-f205326c20f5" Jul 10 19:08:28.685558 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:28.685519 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hgv" Jul 10 19:08:31.324149 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:31.324104 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:31.326774 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:31.326744 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c36cf591-5252-4695-8047-20f5a08d5908-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7g8z\" (UID: \"c36cf591-5252-4695-8047-20f5a08d5908\") " pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:31.415133 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:31.415095 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vnt9v\"" Jul 10 19:08:31.423038 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:31.423005 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l7g8z" Jul 10 19:08:31.559387 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:31.559356 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l7g8z"] Jul 10 19:08:31.562552 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:31.562528 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc36cf591_5252_4695_8047_20f5a08d5908.slice/crio-75bc65d8e2a9916afcb497489ed6ad0a40dbee978d07ceb3682cb6728a117c65 WatchSource:0}: Error finding container 75bc65d8e2a9916afcb497489ed6ad0a40dbee978d07ceb3682cb6728a117c65: Status 404 returned error can't find the container with id 75bc65d8e2a9916afcb497489ed6ad0a40dbee978d07ceb3682cb6728a117c65 Jul 10 19:08:31.842227 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:31.842186 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l7g8z" event={"ID":"c36cf591-5252-4695-8047-20f5a08d5908","Type":"ContainerStarted","Data":"75bc65d8e2a9916afcb497489ed6ad0a40dbee978d07ceb3682cb6728a117c65"} Jul 10 19:08:34.151466 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.151398 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:08:34.154591 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.154565 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 19:08:34.164712 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.164681 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9c4bb811-9e79-4b09-84fe-a6e7debe4cee-original-pull-secret\") pod \"global-pull-secret-syncer-h42d9\" (UID: \"9c4bb811-9e79-4b09-84fe-a6e7debe4cee\") " pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:08:34.252506 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.252456 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:08:34.255457 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.255427 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 19:08:34.265842 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.265815 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feb3ac88-148f-4921-9c03-dbe7f3d3da96-metrics-certs\") pod \"network-metrics-daemon-7ng4r\" (UID: \"feb3ac88-148f-4921-9c03-dbe7f3d3da96\") " pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:08:34.353478 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.353438 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:08:34.356437 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.356408 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btjgd\" (UniqueName: \"kubernetes.io/projected/bf3e3721-db7c-4586-ba94-72732ce64494-kube-api-access-btjgd\") pod \"network-check-target-cn5r4\" (UID: \"bf3e3721-db7c-4586-ba94-72732ce64494\") " pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:08:34.426496 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.426417 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xwfmf\"" Jul 10 19:08:34.432834 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.432803 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-55ch6\"" Jul 10 19:08:34.434849 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.434821 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:08:34.436061 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.436045 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42d9" Jul 10 19:08:34.441311 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:34.441286 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ng4r" Jul 10 19:08:35.263076 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.263043 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:35.263479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.263098 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:35.263479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.263165 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:35.263479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.263220 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:35.263479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.263284 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:35.263479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.263323 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:35.264237 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.263949 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1510cc81-e171-470a-bc51-ab3243a880b7-service-ca-bundle\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:35.267258 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.267208 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"image-registry-6cdc884548-n9267\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:35.268746 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.268700 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9125148c-8264-4c42-bb5a-a811fc4f5588-metrics-tls\") pod \"dns-default-fqbbz\" (UID: \"9125148c-8264-4c42-bb5a-a811fc4f5588\") " pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:35.269843 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.269808 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1510cc81-e171-470a-bc51-ab3243a880b7-metrics-certs\") pod \"router-default-5c65fc6986-nkwgn\" (UID: \"1510cc81-e171-470a-bc51-ab3243a880b7\") " pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:35.269974 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.269810 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/200df66b-8fb3-408f-b765-520943132d83-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-kqtfn\" (UID: \"200df66b-8fb3-408f-b765-520943132d83\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:35.270661 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.270615 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae262dc-c996-4f1d-b118-75619fbd3609-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-bn47g\" (UID: \"dae262dc-c996-4f1d-b118-75619fbd3609\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:35.370346 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.370316 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7ng4r"] Jul 10 19:08:35.378426 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:35.378384 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeb3ac88_148f_4921_9c03_dbe7f3d3da96.slice/crio-21d0afd6d622eb3f36272311608ccf79489f899a6e8c273b33e0ca1bdd19159a WatchSource:0}: Error finding container 21d0afd6d622eb3f36272311608ccf79489f899a6e8c273b33e0ca1bdd19159a: Status 404 returned error can't find the container with id 21d0afd6d622eb3f36272311608ccf79489f899a6e8c273b33e0ca1bdd19159a Jul 10 19:08:35.387813 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.387773 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h42d9"] Jul 10 19:08:35.393571 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:35.393529 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4bb811_9e79_4b09_84fe_a6e7debe4cee.slice/crio-d2f33f5cb08f74cb7ba1bf7cfd79cad701e4325e3cd3ee6a05281d3228acbffb WatchSource:0}: Error finding container d2f33f5cb08f74cb7ba1bf7cfd79cad701e4325e3cd3ee6a05281d3228acbffb: Status 404 returned error can't find the container with id d2f33f5cb08f74cb7ba1bf7cfd79cad701e4325e3cd3ee6a05281d3228acbffb Jul 10 19:08:35.407571 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.407542 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cn5r4"] Jul 10 19:08:35.409607 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:35.409578 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf3e3721_db7c_4586_ba94_72732ce64494.slice/crio-1eeaf8f1d57c86ab9a158ffe8fe0df4465cd0b715190d161166c438da209fbf5 WatchSource:0}: Error finding container 1eeaf8f1d57c86ab9a158ffe8fe0df4465cd0b715190d161166c438da209fbf5: Status 404 returned error can't find the container with id 1eeaf8f1d57c86ab9a158ffe8fe0df4465cd0b715190d161166c438da209fbf5 Jul 10 19:08:35.411133 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.411105 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-f5smk\"" Jul 10 19:08:35.419008 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.418986 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" Jul 10 19:08:35.433638 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.433617 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lqfvm\"" Jul 10 19:08:35.441917 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.441881 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:35.450098 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.449942 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nlsh4\"" Jul 10 19:08:35.459819 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.459579 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" Jul 10 19:08:35.474771 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.474742 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bjqn6\"" Jul 10 19:08:35.483552 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.483518 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:35.495273 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.495216 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-z5rf9\"" Jul 10 19:08:35.504478 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.504011 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:35.512477 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.512082 2556 scope.go:117] "RemoveContainer" containerID="f816faacf32a1d97e1a3c88cf4593b96fd89b66c194b055c26d459e3e37cf7b2" Jul 10 19:08:35.588460 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.587760 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g"] Jul 10 19:08:35.639268 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.639159 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cdc884548-n9267"] Jul 10 19:08:35.678599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.678563 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn"] Jul 10 19:08:35.715758 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.715630 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c65fc6986-nkwgn"] Jul 10 19:08:35.760231 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.760124 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fqbbz"] Jul 10 19:08:35.763558 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:08:35.763523 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9125148c_8264_4c42_bb5a_a811fc4f5588.slice/crio-5e36e7fa20f3ec129c49af8768e00e392a148ba879e6a21c938b7637a316f1ba WatchSource:0}: Error finding container 5e36e7fa20f3ec129c49af8768e00e392a148ba879e6a21c938b7637a316f1ba: Status 404 returned error can't find the container with id 5e36e7fa20f3ec129c49af8768e00e392a148ba879e6a21c938b7637a316f1ba Jul 10 19:08:35.860071 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.860031 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7ng4r" event={"ID":"feb3ac88-148f-4921-9c03-dbe7f3d3da96","Type":"ContainerStarted","Data":"21d0afd6d622eb3f36272311608ccf79489f899a6e8c273b33e0ca1bdd19159a"} Jul 10 19:08:35.861799 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.861756 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c65fc6986-nkwgn" event={"ID":"1510cc81-e171-470a-bc51-ab3243a880b7","Type":"ContainerStarted","Data":"9fe25f35b5403f39ba10102237db9935db412b3b079e71f1aaebb7f2665d528f"} Jul 10 19:08:35.861979 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.861820 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c65fc6986-nkwgn" event={"ID":"1510cc81-e171-470a-bc51-ab3243a880b7","Type":"ContainerStarted","Data":"75de2f2c81f8f7bebeaf25fc14e553254462dadc7a4e9ea420d62a9220318aae"} Jul 10 19:08:35.863111 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.863083 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" event={"ID":"200df66b-8fb3-408f-b765-520943132d83","Type":"ContainerStarted","Data":"fd4569977dcc27382aaecd437a71fcf57ad1a84db067e1ebd6b6d0a2ebdf2500"} Jul 10 19:08:35.864600 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.864579 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l7g8z" event={"ID":"c36cf591-5252-4695-8047-20f5a08d5908","Type":"ContainerStarted","Data":"2842f13e9e8cb8e82d42e5895fa1ab0cdf5c6289c2ad6d87b3fc11164ab87926"} Jul 10 19:08:35.866000 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.865973 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqbbz" event={"ID":"9125148c-8264-4c42-bb5a-a811fc4f5588","Type":"ContainerStarted","Data":"5e36e7fa20f3ec129c49af8768e00e392a148ba879e6a21c938b7637a316f1ba"} Jul 10 19:08:35.867239 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.867219 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" event={"ID":"dae262dc-c996-4f1d-b118-75619fbd3609","Type":"ContainerStarted","Data":"b71b9ca19794dbf3a6ff589559eec1403577ba91b5d471a4ace4f477b123646c"} Jul 10 19:08:35.868744 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.868703 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cn5r4" event={"ID":"bf3e3721-db7c-4586-ba94-72732ce64494","Type":"ContainerStarted","Data":"e22eff66ca9a02a0a530e2bf81af9667560be90c43c4f2f592319e2923dbb6ba"} Jul 10 19:08:35.868853 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.868762 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cn5r4" event={"ID":"bf3e3721-db7c-4586-ba94-72732ce64494","Type":"ContainerStarted","Data":"1eeaf8f1d57c86ab9a158ffe8fe0df4465cd0b715190d161166c438da209fbf5"} Jul 10 19:08:35.868853 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.868811 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:08:35.870725 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.870704 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:08:35.871363 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.870806 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-gxdsf" event={"ID":"633078d5-6344-4d3c-b710-f205326c20f5","Type":"ContainerStarted","Data":"b04c31b4266eab99703b8ff4298738a2ee942acb33f9b6fa9b90cc4b0dc22fa7"} Jul 10 19:08:35.871363 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.871137 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:35.872600 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.872578 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdc884548-n9267" event={"ID":"1a7e3559-1409-44cf-b46a-af8e91f48f75","Type":"ContainerStarted","Data":"0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361"} Jul 10 19:08:35.872767 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.872748 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdc884548-n9267" event={"ID":"1a7e3559-1409-44cf-b46a-af8e91f48f75","Type":"ContainerStarted","Data":"db37a179186d24914b1eb288b5fd8809c96a9c2ae65dccc774d43d8899f6c039"} Jul 10 19:08:35.872925 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.872902 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:08:35.873786 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.873758 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h42d9" event={"ID":"9c4bb811-9e79-4b09-84fe-a6e7debe4cee","Type":"ContainerStarted","Data":"d2f33f5cb08f74cb7ba1bf7cfd79cad701e4325e3cd3ee6a05281d3228acbffb"} Jul 10 19:08:35.880272 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.880218 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5c65fc6986-nkwgn" podStartSLOduration=39.880200061 podStartE2EDuration="39.880200061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:35.879776435 +0000 UTC m=+67.896234806" watchObservedRunningTime="2026-07-10 19:08:35.880200061 +0000 UTC m=+67.896658434" Jul 10 19:08:35.896390 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.896329 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-gxdsf" podStartSLOduration=30.521061781 podStartE2EDuration="39.896304432s" totalImagesPullingTime="9.375242651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:35.895310453 +0000 UTC m=+67.911768825" watchObservedRunningTime="2026-07-10 19:08:35.896304432 +0000 UTC m=+67.912762804" Jul 10 19:08:35.910061 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:35.909998 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cn5r4" podStartSLOduration=66.90997573 podStartE2EDuration="1m6.90997573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:35.909818664 +0000 UTC m=+67.926277036" watchObservedRunningTime="2026-07-10 19:08:35.90997573 +0000 UTC m=+67.926434102" Jul 10 19:08:36.139480 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:36.139148 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-gxdsf" Jul 10 19:08:36.162152 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:36.162087 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cdc884548-n9267" podStartSLOduration=67.162064759 podStartE2EDuration="1m7.162064759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:35.927938159 +0000 UTC m=+67.944396533" watchObservedRunningTime="2026-07-10 19:08:36.162064759 +0000 UTC m=+68.178523131" Jul 10 19:08:36.484853 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:36.484775 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:36.488016 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:36.487768 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:36.884746 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:36.884707 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l7g8z" event={"ID":"c36cf591-5252-4695-8047-20f5a08d5908","Type":"ContainerStarted","Data":"89155a8ba9d773f5e3c617505433e8ed62412a608552c7e7009800616e512c80"} Jul 10 19:08:36.887363 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:36.887300 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:36.887809 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:36.887607 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5c65fc6986-nkwgn" Jul 10 19:08:37.892354 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:37.891903 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7ng4r" event={"ID":"feb3ac88-148f-4921-9c03-dbe7f3d3da96","Type":"ContainerStarted","Data":"b12605abfff0e1134247f99d91a6bc01a4af6ecb58153912ada55f89442cb992"} Jul 10 19:08:42.914236 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:42.914141 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" event={"ID":"200df66b-8fb3-408f-b765-520943132d83","Type":"ContainerStarted","Data":"b952dea0e4e037245179b23cbec32b7a6b360b9aaae8615a3f74041f0bc6eb44"} Jul 10 19:08:42.917419 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:42.917387 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l7g8z" event={"ID":"c36cf591-5252-4695-8047-20f5a08d5908","Type":"ContainerStarted","Data":"86399681d46479d2f779cd139410ee34faf45fad3e76a91dab7cd75ea868f29f"} Jul 10 19:08:42.919196 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:42.919167 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqbbz" event={"ID":"9125148c-8264-4c42-bb5a-a811fc4f5588","Type":"ContainerStarted","Data":"443d70d204920ff4f68a8385bd8244fab64bfb7ee0ae7efbf4e01ad9ce575bfd"} Jul 10 19:08:42.922326 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:42.921485 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" event={"ID":"dae262dc-c996-4f1d-b118-75619fbd3609","Type":"ContainerStarted","Data":"3005a04fd520c1a30cea4e340519b6092b957459ad33a97a857881cf430270c1"} Jul 10 19:08:42.926753 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:42.926728 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7ng4r" event={"ID":"feb3ac88-148f-4921-9c03-dbe7f3d3da96","Type":"ContainerStarted","Data":"247b33f5b416e8506767bf704e646e1abf8f8a19b1fff484f7883176325a2f79"} Jul 10 19:08:42.972982 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:42.972917 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-l7g8z" podStartSLOduration=17.122649069 podStartE2EDuration="27.972862005s" totalImagesPullingTime="10.850212936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:08:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:42.970978243 +0000 UTC m=+74.987436614" watchObservedRunningTime="2026-07-10 19:08:42.972862005 +0000 UTC m=+74.989320380" Jul 10 19:08:43.016667 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:43.016603 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7ng4r" podStartSLOduration=72.137573888 podStartE2EDuration="1m14.016579523s" totalImagesPullingTime="1.879005635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:43.016268553 +0000 UTC m=+75.032726915" watchObservedRunningTime="2026-07-10 19:08:43.016579523 +0000 UTC m=+75.033037897" Jul 10 19:08:43.051833 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:43.051602 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-bn47g" podStartSLOduration=40.044533078 podStartE2EDuration="47.051577399s" totalImagesPullingTime="7.007044321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:43.050095756 +0000 UTC m=+75.066554128" watchObservedRunningTime="2026-07-10 19:08:43.051577399 +0000 UTC m=+75.068035771" Jul 10 19:08:43.932558 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:43.932519 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" event={"ID":"200df66b-8fb3-408f-b765-520943132d83","Type":"ContainerStarted","Data":"f67384ececbb273959b7ed514455f0b0edbe537cbb7f34f5331b0e41f3533599"} Jul 10 19:08:43.934442 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:43.934416 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqbbz" event={"ID":"9125148c-8264-4c42-bb5a-a811fc4f5588","Type":"ContainerStarted","Data":"ac41eb915c7e6d33641804ca2e72d05179f74b849e77f84a4c111e6c4669518c"} Jul 10 19:08:43.934576 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:43.934502 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:43.935808 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:43.935787 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h42d9" event={"ID":"9c4bb811-9e79-4b09-84fe-a6e7debe4cee","Type":"ContainerStarted","Data":"96632d008190ed4190eb29260228f84c10091afbbdcdce6f5ab55478a4d61d9e"} Jul 10 19:08:43.951420 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:43.951375 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-kqtfn" podStartSLOduration=41.086244592 podStartE2EDuration="47.951356912s" totalImagesPullingTime="6.86511232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:43.94981387 +0000 UTC m=+75.966272260" watchObservedRunningTime="2026-07-10 19:08:43.951356912 +0000 UTC m=+75.967815282" Jul 10 19:08:43.970934 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:43.970862 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fqbbz" podStartSLOduration=34.131266082 podStartE2EDuration="40.970848408s" totalImagesPullingTime="6.839582326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:43.969704992 +0000 UTC m=+75.986163364" watchObservedRunningTime="2026-07-10 19:08:43.970848408 +0000 UTC m=+75.987306779" Jul 10 19:08:43.990842 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:43.990796 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h42d9" podStartSLOduration=67.455787281 podStartE2EDuration="1m14.990778991s" totalImagesPullingTime="7.53499171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:08:43.990414838 +0000 UTC m=+76.006873210" watchObservedRunningTime="2026-07-10 19:08:43.990778991 +0000 UTC m=+76.007237362" Jul 10 19:08:53.942283 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:53.942173 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fqbbz" Jul 10 19:08:55.446296 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:55.446249 2556 patch_prober.go:28] interesting pod/image-registry-6cdc884548-n9267 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 19:08:55.446670 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:55.446308 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cdc884548-n9267" podUID="1a7e3559-1409-44cf-b46a-af8e91f48f75" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 19:08:56.890382 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:08:56.890352 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:09:03.827078 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.827040 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-2h5dm"] Jul 10 19:09:03.830446 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.830425 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:03.833180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.833153 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 10 19:09:03.833316 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.833215 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 10 19:09:03.834092 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.834073 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 10 19:09:03.834158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.834080 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 19:09:03.834343 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.834326 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8b7jf\"" Jul 10 19:09:03.842705 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.842435 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-797tj"] Jul 10 19:09:03.847368 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.847337 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-2h5dm"] Jul 10 19:09:03.847493 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.847456 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:03.850195 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.850173 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 19:09:03.850338 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.850204 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 19:09:03.850338 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.850282 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cn7z6\"" Jul 10 19:09:03.850462 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:03.850284 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 19:09:04.011445 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011400 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.011445 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011445 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3936321e-af64-4cdc-939a-db6f017e7459-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.011714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011476 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-tls\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.011714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011499 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-wtmp\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.011714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011524 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3936321e-af64-4cdc-939a-db6f017e7459-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.011714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011547 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-textfile\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.011714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011568 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c72933f-91a8-434e-9984-9fce7ef47035-metrics-client-ca\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.011714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011642 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3936321e-af64-4cdc-939a-db6f017e7459-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.011714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011665 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk5tm\" (UniqueName: \"kubernetes.io/projected/8c72933f-91a8-434e-9984-9fce7ef47035-kube-api-access-jk5tm\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.011714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011701 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c72933f-91a8-434e-9984-9fce7ef47035-sys\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.012135 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011724 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-accelerators-collector-config\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.012135 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011751 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3936321e-af64-4cdc-939a-db6f017e7459-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.012135 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011779 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c72933f-91a8-434e-9984-9fce7ef47035-root\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.012135 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011821 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5twb\" (UniqueName: \"kubernetes.io/projected/3936321e-af64-4cdc-939a-db6f017e7459-kube-api-access-b5twb\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.012135 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.011916 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3936321e-af64-4cdc-939a-db6f017e7459-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.102824 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.102741 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cdc884548-n9267"] Jul 10 19:09:04.112459 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112426 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3936321e-af64-4cdc-939a-db6f017e7459-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.112620 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112470 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.112620 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112489 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3936321e-af64-4cdc-939a-db6f017e7459-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.112620 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112511 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-tls\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.112620 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112529 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-wtmp\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.112620 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3936321e-af64-4cdc-939a-db6f017e7459-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.112620 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112566 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-textfile\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.112620 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112582 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c72933f-91a8-434e-9984-9fce7ef47035-metrics-client-ca\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.112620 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112610 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3936321e-af64-4cdc-939a-db6f017e7459-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.113023 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112625 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk5tm\" (UniqueName: \"kubernetes.io/projected/8c72933f-91a8-434e-9984-9fce7ef47035-kube-api-access-jk5tm\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.113023 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112644 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c72933f-91a8-434e-9984-9fce7ef47035-sys\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.113023 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112660 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-accelerators-collector-config\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.113023 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112678 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3936321e-af64-4cdc-939a-db6f017e7459-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.113023 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112696 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c72933f-91a8-434e-9984-9fce7ef47035-root\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.113023 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112720 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5twb\" (UniqueName: \"kubernetes.io/projected/3936321e-af64-4cdc-939a-db6f017e7459-kube-api-access-b5twb\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.113023 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.112785 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c72933f-91a8-434e-9984-9fce7ef47035-sys\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.113363 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.113080 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c72933f-91a8-434e-9984-9fce7ef47035-root\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.113489 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.113447 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c72933f-91a8-434e-9984-9fce7ef47035-metrics-client-ca\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.113629 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.113543 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-accelerators-collector-config\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.113829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.113757 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3936321e-af64-4cdc-939a-db6f017e7459-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.114522 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.114494 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-wtmp\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.114727 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.114705 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-textfile\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.114850 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.114825 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3936321e-af64-4cdc-939a-db6f017e7459-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.115665 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.115274 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3936321e-af64-4cdc-939a-db6f017e7459-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.116330 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.116055 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3936321e-af64-4cdc-939a-db6f017e7459-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.116449 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.116349 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.116948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.116922 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3936321e-af64-4cdc-939a-db6f017e7459-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.117438 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.117416 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c72933f-91a8-434e-9984-9fce7ef47035-node-exporter-tls\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.128357 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.128272 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk5tm\" (UniqueName: \"kubernetes.io/projected/8c72933f-91a8-434e-9984-9fce7ef47035-kube-api-access-jk5tm\") pod \"node-exporter-797tj\" (UID: \"8c72933f-91a8-434e-9984-9fce7ef47035\") " pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.128576 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.128402 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5twb\" (UniqueName: \"kubernetes.io/projected/3936321e-af64-4cdc-939a-db6f017e7459-kube-api-access-b5twb\") pod \"kube-state-metrics-785778ff74-2h5dm\" (UID: \"3936321e-af64-4cdc-939a-db6f017e7459\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.145777 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.145744 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" Jul 10 19:09:04.160659 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.160622 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-797tj" Jul 10 19:09:04.178174 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:09:04.178139 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c72933f_91a8_434e_9984_9fce7ef47035.slice/crio-6ec960a954877675e227bea6d580d4c209720bf5384a3f6e26825c04430a0777 WatchSource:0}: Error finding container 6ec960a954877675e227bea6d580d4c209720bf5384a3f6e26825c04430a0777: Status 404 returned error can't find the container with id 6ec960a954877675e227bea6d580d4c209720bf5384a3f6e26825c04430a0777 Jul 10 19:09:04.297515 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:04.297480 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-2h5dm"] Jul 10 19:09:04.300634 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:09:04.300602 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3936321e_af64_4cdc_939a_db6f017e7459.slice/crio-0e4701412963c9bcc4b08b74cac41de6bc190536ad306b27a29c85ac78047891 WatchSource:0}: Error finding container 0e4701412963c9bcc4b08b74cac41de6bc190536ad306b27a29c85ac78047891: Status 404 returned error can't find the container with id 0e4701412963c9bcc4b08b74cac41de6bc190536ad306b27a29c85ac78047891 Jul 10 19:09:05.014507 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:05.014402 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-797tj" event={"ID":"8c72933f-91a8-434e-9984-9fce7ef47035","Type":"ContainerStarted","Data":"6ec960a954877675e227bea6d580d4c209720bf5384a3f6e26825c04430a0777"} Jul 10 19:09:05.016691 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:05.016654 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" event={"ID":"3936321e-af64-4cdc-939a-db6f017e7459","Type":"ContainerStarted","Data":"0e4701412963c9bcc4b08b74cac41de6bc190536ad306b27a29c85ac78047891"} Jul 10 19:09:06.889105 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:06.889074 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cn5r4" Jul 10 19:09:07.032003 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:07.031940 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" event={"ID":"3936321e-af64-4cdc-939a-db6f017e7459","Type":"ContainerStarted","Data":"f5d6ac302b1073a1023e593839980acc00d988681b597b50c22539eeced3a2f3"} Jul 10 19:09:07.032197 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:07.032087 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" event={"ID":"3936321e-af64-4cdc-939a-db6f017e7459","Type":"ContainerStarted","Data":"6660deeef361e5ed8f49785a3983b327a807b38e5cda242ef8520abd73028001"} Jul 10 19:09:07.032197 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:07.032157 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" event={"ID":"3936321e-af64-4cdc-939a-db6f017e7459","Type":"ContainerStarted","Data":"dcbb55ff17053c13eb30da792441cd3ad8a9fd097def55ec35681753ecb8e2d8"} Jul 10 19:09:07.033848 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:07.033812 2556 generic.go:356] "Generic (PLEG): container finished" podID="8c72933f-91a8-434e-9984-9fce7ef47035" containerID="843e8314e24b3bfea78d90d5e4fccb668a54737259ec2f71dd6df6b3a1d0bafd" exitCode=0 Jul 10 19:09:07.033978 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:07.033904 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-797tj" event={"ID":"8c72933f-91a8-434e-9984-9fce7ef47035","Type":"ContainerDied","Data":"843e8314e24b3bfea78d90d5e4fccb668a54737259ec2f71dd6df6b3a1d0bafd"} Jul 10 19:09:07.053337 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:07.053276 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-2h5dm" podStartSLOduration=2.34580254 podStartE2EDuration="4.053255709s" totalImagesPullingTime="1.707453169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:09:07.052029033 +0000 UTC m=+99.068487396" watchObservedRunningTime="2026-07-10 19:09:07.053255709 +0000 UTC m=+99.069714081" Jul 10 19:09:08.040161 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:08.040125 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-797tj" event={"ID":"8c72933f-91a8-434e-9984-9fce7ef47035","Type":"ContainerStarted","Data":"54b8843d04abd0d32e74e85e25785cdf7e208ad81766427d0b9988975412a053"} Jul 10 19:09:08.040581 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:08.040180 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-797tj" event={"ID":"8c72933f-91a8-434e-9984-9fce7ef47035","Type":"ContainerStarted","Data":"00d980210344627a5f0e659c960588bbb17778ac1d8d239066ecf0abaff278de"} Jul 10 19:09:08.060361 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:08.060306 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-797tj" podStartSLOduration=3.149235751 podStartE2EDuration="5.060286851s" totalImagesPullingTime="1.800213651s" totalInitContainerRuntime="110.837449ms" isStatefulPod=true podCreationTimestamp="2026-07-10 19:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:09:08.058109279 +0000 UTC m=+100.074567677" watchObservedRunningTime="2026-07-10 19:09:08.060286851 +0000 UTC m=+100.076745222" Jul 10 19:09:09.243211 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.243172 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5785f94766-d5x62"] Jul 10 19:09:09.246421 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.246394 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.248960 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.248928 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 19:09:09.248960 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.248949 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 19:09:09.249182 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.249083 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 19:09:09.250370 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.250151 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 19:09:09.250370 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.250283 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 19:09:09.250649 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.250437 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9xtw5\"" Jul 10 19:09:09.250649 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.250530 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 19:09:09.250649 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.250529 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 19:09:09.257012 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.256973 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-trusted-ca-bundle\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.257456 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.257022 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-serving-cert\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.257456 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.257050 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jwrb\" (UniqueName: \"kubernetes.io/projected/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-kube-api-access-8jwrb\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.257456 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.257108 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-config\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.257456 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.257218 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-oauth-config\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.257456 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.257273 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-service-ca\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.257456 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.257358 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-oauth-serving-cert\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.259892 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.259354 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 19:09:09.260538 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.260512 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5785f94766-d5x62"] Jul 10 19:09:09.357965 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.357924 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-config\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.358161 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.357976 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-oauth-config\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.358161 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.357991 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-service-ca\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.358161 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.358024 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-oauth-serving-cert\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.358161 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.358049 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-trusted-ca-bundle\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.358161 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.358067 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-serving-cert\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.358161 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.358083 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jwrb\" (UniqueName: \"kubernetes.io/projected/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-kube-api-access-8jwrb\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.358799 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.358769 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-service-ca\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.358799 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.358791 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-oauth-serving-cert\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.358975 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.358926 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-trusted-ca-bundle\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.359153 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.359137 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-config\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.360694 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.360677 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-serving-cert\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.360748 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.360722 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-oauth-config\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.367497 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.367469 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jwrb\" (UniqueName: \"kubernetes.io/projected/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-kube-api-access-8jwrb\") pod \"console-5785f94766-d5x62\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.563448 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.563355 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:09.707417 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:09.707392 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5785f94766-d5x62"] Jul 10 19:09:09.709513 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:09:09.709485 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5604af00_83fa_4edc_b258_ae6cc3f8bbc9.slice/crio-9111d5cc7c0f03d12ae680641cb67222eb3786d772f731f50a2cca2b04769b79 WatchSource:0}: Error finding container 9111d5cc7c0f03d12ae680641cb67222eb3786d772f731f50a2cca2b04769b79: Status 404 returned error can't find the container with id 9111d5cc7c0f03d12ae680641cb67222eb3786d772f731f50a2cca2b04769b79 Jul 10 19:09:10.049657 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:10.049621 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5785f94766-d5x62" event={"ID":"5604af00-83fa-4edc-b258-ae6cc3f8bbc9","Type":"ContainerStarted","Data":"9111d5cc7c0f03d12ae680641cb67222eb3786d772f731f50a2cca2b04769b79"} Jul 10 19:09:14.067270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:14.067227 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5785f94766-d5x62" event={"ID":"5604af00-83fa-4edc-b258-ae6cc3f8bbc9","Type":"ContainerStarted","Data":"544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c"} Jul 10 19:09:14.085451 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:14.085391 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5785f94766-d5x62" podStartSLOduration=1.6596728010000001 podStartE2EDuration="5.085375994s" totalImagesPullingTime="3.425703193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:09:14.084482641 +0000 UTC m=+106.100941013" watchObservedRunningTime="2026-07-10 19:09:14.085375994 +0000 UTC m=+106.101834367" Jul 10 19:09:19.564302 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:19.564264 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:19.564783 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:19.564340 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:19.569457 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:19.569430 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:20.093150 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:20.093119 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:09:25.108736 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:25.108702 2556 generic.go:356] "Generic (PLEG): container finished" podID="65b055a9-5e50-44ee-9e0a-000735040140" containerID="1e5361aa86bb58c5637b026a4d4556d5e87389556f06627e1a8ff11f9c7c0122" exitCode=0 Jul 10 19:09:25.109294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:25.108783 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" event={"ID":"65b055a9-5e50-44ee-9e0a-000735040140","Type":"ContainerDied","Data":"1e5361aa86bb58c5637b026a4d4556d5e87389556f06627e1a8ff11f9c7c0122"} Jul 10 19:09:25.109294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:25.109220 2556 scope.go:117] "RemoveContainer" containerID="1e5361aa86bb58c5637b026a4d4556d5e87389556f06627e1a8ff11f9c7c0122" Jul 10 19:09:25.110377 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:25.110349 2556 generic.go:356] "Generic (PLEG): container finished" podID="324a3506-415f-46fc-94f0-47029379437d" containerID="176e809e96c6d92ab990ac4bb91bc56c1d4c300c38c6aef49ef364d88aa1728d" exitCode=0 Jul 10 19:09:25.110472 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:25.110410 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" event={"ID":"324a3506-415f-46fc-94f0-47029379437d","Type":"ContainerDied","Data":"176e809e96c6d92ab990ac4bb91bc56c1d4c300c38c6aef49ef364d88aa1728d"} Jul 10 19:09:25.110743 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:25.110727 2556 scope.go:117] "RemoveContainer" containerID="176e809e96c6d92ab990ac4bb91bc56c1d4c300c38c6aef49ef364d88aa1728d" Jul 10 19:09:26.116012 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:26.115976 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pzvv2" event={"ID":"65b055a9-5e50-44ee-9e0a-000735040140","Type":"ContainerStarted","Data":"80ca8de153abb1bd4635aaf73466caf7db635c95067eb66228f2f0506c0832d3"} Jul 10 19:09:26.117799 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:26.117772 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-gqxwz" event={"ID":"324a3506-415f-46fc-94f0-47029379437d","Type":"ContainerStarted","Data":"9a4fa8adb3ab95ac922e07a13a2f8478f9e71f65ed9a9d0e3991cad5017e1167"} Jul 10 19:09:29.128753 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.128686 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cdc884548-n9267" podUID="1a7e3559-1409-44cf-b46a-af8e91f48f75" containerName="registry" containerID="cri-o://0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361" gracePeriod=30 Jul 10 19:09:29.387731 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.387664 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:09:29.549838 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.549794 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-bound-sa-token\") pod \"1a7e3559-1409-44cf-b46a-af8e91f48f75\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " Jul 10 19:09:29.550051 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.549861 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-trusted-ca\") pod \"1a7e3559-1409-44cf-b46a-af8e91f48f75\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " Jul 10 19:09:29.550051 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.549934 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-certificates\") pod \"1a7e3559-1409-44cf-b46a-af8e91f48f75\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " Jul 10 19:09:29.550126 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.550053 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-image-registry-private-configuration\") pod \"1a7e3559-1409-44cf-b46a-af8e91f48f75\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " Jul 10 19:09:29.550126 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.550077 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-installation-pull-secrets\") pod \"1a7e3559-1409-44cf-b46a-af8e91f48f75\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " Jul 10 19:09:29.550126 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.550101 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbp25\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-kube-api-access-qbp25\") pod \"1a7e3559-1409-44cf-b46a-af8e91f48f75\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " Jul 10 19:09:29.550266 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.550133 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a7e3559-1409-44cf-b46a-af8e91f48f75-ca-trust-extracted\") pod \"1a7e3559-1409-44cf-b46a-af8e91f48f75\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " Jul 10 19:09:29.550266 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.550179 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") pod \"1a7e3559-1409-44cf-b46a-af8e91f48f75\" (UID: \"1a7e3559-1409-44cf-b46a-af8e91f48f75\") " Jul 10 19:09:29.550362 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.550342 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1a7e3559-1409-44cf-b46a-af8e91f48f75" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:09:29.550417 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.550385 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1a7e3559-1409-44cf-b46a-af8e91f48f75" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:09:29.550586 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.550565 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-trusted-ca\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:09:29.550663 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.550590 2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-certificates\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:09:29.552588 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.552555 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1a7e3559-1409-44cf-b46a-af8e91f48f75" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:09:29.552749 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.552624 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1a7e3559-1409-44cf-b46a-af8e91f48f75" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:09:29.552999 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.552967 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1a7e3559-1409-44cf-b46a-af8e91f48f75" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:09:29.553139 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.553115 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1a7e3559-1409-44cf-b46a-af8e91f48f75" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:09:29.553184 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.553163 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-kube-api-access-qbp25" (OuterVolumeSpecName: "kube-api-access-qbp25") pod "1a7e3559-1409-44cf-b46a-af8e91f48f75" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75"). InnerVolumeSpecName "kube-api-access-qbp25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:09:29.558828 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.558792 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a7e3559-1409-44cf-b46a-af8e91f48f75-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1a7e3559-1409-44cf-b46a-af8e91f48f75" (UID: "1a7e3559-1409-44cf-b46a-af8e91f48f75"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:09:29.651263 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.651162 2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-registry-tls\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:09:29.651263 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.651206 2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-bound-sa-token\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:09:29.651263 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.651216 2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-image-registry-private-configuration\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:09:29.651263 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.651226 2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1a7e3559-1409-44cf-b46a-af8e91f48f75-installation-pull-secrets\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:09:29.651263 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.651236 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbp25\" (UniqueName: \"kubernetes.io/projected/1a7e3559-1409-44cf-b46a-af8e91f48f75-kube-api-access-qbp25\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:09:29.651263 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:29.651245 2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1a7e3559-1409-44cf-b46a-af8e91f48f75-ca-trust-extracted\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:09:30.141070 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.141023 2556 generic.go:356] "Generic (PLEG): container finished" podID="1a7e3559-1409-44cf-b46a-af8e91f48f75" containerID="0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361" exitCode=0 Jul 10 19:09:30.141070 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.141075 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdc884548-n9267" Jul 10 19:09:30.141544 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.141107 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdc884548-n9267" event={"ID":"1a7e3559-1409-44cf-b46a-af8e91f48f75","Type":"ContainerDied","Data":"0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361"} Jul 10 19:09:30.141544 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.141168 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdc884548-n9267" event={"ID":"1a7e3559-1409-44cf-b46a-af8e91f48f75","Type":"ContainerDied","Data":"db37a179186d24914b1eb288b5fd8809c96a9c2ae65dccc774d43d8899f6c039"} Jul 10 19:09:30.141544 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.141190 2556 scope.go:117] "RemoveContainer" containerID="0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361" Jul 10 19:09:30.150368 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.150352 2556 scope.go:117] "RemoveContainer" containerID="0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361" Jul 10 19:09:30.150724 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:09:30.150703 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361\": container with ID starting with 0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361 not found: ID does not exist" containerID="0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361" Jul 10 19:09:30.150767 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.150734 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361"} err="failed to get container status \"0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361\": rpc error: code = NotFound desc = could not find container \"0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361\": container with ID starting with 0d1e3c2c8f56e2f43822c6ea6957f96d39c2f43c523dc611676349ac31f94361 not found: ID does not exist" Jul 10 19:09:30.161911 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.161858 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cdc884548-n9267"] Jul 10 19:09:30.165553 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.165526 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cdc884548-n9267"] Jul 10 19:09:30.516325 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:30.516284 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a7e3559-1409-44cf-b46a-af8e91f48f75" path="/var/lib/kubelet/pods/1a7e3559-1409-44cf-b46a-af8e91f48f75/volumes" Jul 10 19:09:45.205277 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:45.205237 2556 generic.go:356] "Generic (PLEG): container finished" podID="2324f695-e128-4d57-950e-af3f5c363986" containerID="260563b01365c8cfcb51fca9702bc3f53ea71c237309fa57f6153aa1a76acf1e" exitCode=0 Jul 10 19:09:45.205812 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:45.205316 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" event={"ID":"2324f695-e128-4d57-950e-af3f5c363986","Type":"ContainerDied","Data":"260563b01365c8cfcb51fca9702bc3f53ea71c237309fa57f6153aa1a76acf1e"} Jul 10 19:09:45.205812 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:45.205753 2556 scope.go:117] "RemoveContainer" containerID="260563b01365c8cfcb51fca9702bc3f53ea71c237309fa57f6153aa1a76acf1e" Jul 10 19:09:46.211281 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:09:46.211248 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7h4ht" event={"ID":"2324f695-e128-4d57-950e-af3f5c363986","Type":"ContainerStarted","Data":"93ac241262a768a56941dbcdde116e7718655f4f62d4d6919df9082eeca5c7e8"} Jul 10 19:10:32.378500 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.378467 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f66cc496f-dh5j4"] Jul 10 19:10:32.378979 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.378778 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a7e3559-1409-44cf-b46a-af8e91f48f75" containerName="registry" Jul 10 19:10:32.378979 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.378790 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a7e3559-1409-44cf-b46a-af8e91f48f75" containerName="registry" Jul 10 19:10:32.378979 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.378853 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a7e3559-1409-44cf-b46a-af8e91f48f75" containerName="registry" Jul 10 19:10:32.383559 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.383530 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.392697 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.392665 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f66cc496f-dh5j4"] Jul 10 19:10:32.473564 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.473519 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-oauth-config\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.473564 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.473558 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-console-config\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.473806 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.473585 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-trusted-ca-bundle\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.473806 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.473606 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-oauth-serving-cert\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.473806 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.473733 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-serving-cert\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.473806 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.473788 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wcwc\" (UniqueName: \"kubernetes.io/projected/da552e32-fe26-456e-92c8-712e87a7314c-kube-api-access-6wcwc\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.474026 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.473810 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-service-ca\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.574984 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.574946 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-oauth-config\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.574984 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.574987 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-console-config\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.575253 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.575014 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-trusted-ca-bundle\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.575253 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.575040 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-oauth-serving-cert\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.575253 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.575210 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-serving-cert\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.575253 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.575247 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wcwc\" (UniqueName: \"kubernetes.io/projected/da552e32-fe26-456e-92c8-712e87a7314c-kube-api-access-6wcwc\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.575451 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.575279 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-service-ca\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.575835 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.575806 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-console-config\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.575989 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.575946 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-trusted-ca-bundle\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.576044 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.576007 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-oauth-serving-cert\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.576260 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.576236 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-service-ca\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.577693 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.577668 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-oauth-config\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.577941 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.577924 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-serving-cert\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.582851 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.582823 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wcwc\" (UniqueName: \"kubernetes.io/projected/da552e32-fe26-456e-92c8-712e87a7314c-kube-api-access-6wcwc\") pod \"console-5f66cc496f-dh5j4\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.694945 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.694824 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:32.836682 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:32.836649 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f66cc496f-dh5j4"] Jul 10 19:10:32.840068 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:10:32.840039 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda552e32_fe26_456e_92c8_712e87a7314c.slice/crio-62a8b11bd78dc09de0a6d1a5193a3b4d9d7e10cc48e30d94c84f1f98d2967b5e WatchSource:0}: Error finding container 62a8b11bd78dc09de0a6d1a5193a3b4d9d7e10cc48e30d94c84f1f98d2967b5e: Status 404 returned error can't find the container with id 62a8b11bd78dc09de0a6d1a5193a3b4d9d7e10cc48e30d94c84f1f98d2967b5e Jul 10 19:10:33.392472 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:33.392431 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f66cc496f-dh5j4" event={"ID":"da552e32-fe26-456e-92c8-712e87a7314c","Type":"ContainerStarted","Data":"5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37"} Jul 10 19:10:33.392843 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:33.392485 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f66cc496f-dh5j4" event={"ID":"da552e32-fe26-456e-92c8-712e87a7314c","Type":"ContainerStarted","Data":"62a8b11bd78dc09de0a6d1a5193a3b4d9d7e10cc48e30d94c84f1f98d2967b5e"} Jul 10 19:10:33.410495 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:33.410435 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f66cc496f-dh5j4" podStartSLOduration=1.410414168 podStartE2EDuration="1.410414168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:10:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:10:33.408542038 +0000 UTC m=+185.425000515" watchObservedRunningTime="2026-07-10 19:10:33.410414168 +0000 UTC m=+185.426872538" Jul 10 19:10:42.695464 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:42.695425 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:42.695987 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:42.695491 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:42.700585 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:42.700554 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:43.434179 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:43.434139 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:10:43.486967 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:10:43.486930 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5785f94766-d5x62"] Jul 10 19:11:08.516523 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.516485 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5785f94766-d5x62" podUID="5604af00-83fa-4edc-b258-ae6cc3f8bbc9" containerName="console" containerID="cri-o://544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c" gracePeriod=15 Jul 10 19:11:08.761282 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.761255 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:11:08.858944 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.858813 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-service-ca\") pod \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " Jul 10 19:11:08.858944 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.858864 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-oauth-serving-cert\") pod \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " Jul 10 19:11:08.859180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859009 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-trusted-ca-bundle\") pod \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " Jul 10 19:11:08.859180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859051 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-serving-cert\") pod \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " Jul 10 19:11:08.859180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859068 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-oauth-config\") pod \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " Jul 10 19:11:08.859180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859136 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-config\") pod \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " Jul 10 19:11:08.859180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859178 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jwrb\" (UniqueName: \"kubernetes.io/projected/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-kube-api-access-8jwrb\") pod \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\" (UID: \"5604af00-83fa-4edc-b258-ae6cc3f8bbc9\") " Jul 10 19:11:08.859429 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859369 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5604af00-83fa-4edc-b258-ae6cc3f8bbc9" (UID: "5604af00-83fa-4edc-b258-ae6cc3f8bbc9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:11:08.859429 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859380 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-service-ca" (OuterVolumeSpecName: "service-ca") pod "5604af00-83fa-4edc-b258-ae6cc3f8bbc9" (UID: "5604af00-83fa-4edc-b258-ae6cc3f8bbc9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:11:08.859429 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859418 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5604af00-83fa-4edc-b258-ae6cc3f8bbc9" (UID: "5604af00-83fa-4edc-b258-ae6cc3f8bbc9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:11:08.859592 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859566 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-service-ca\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:11:08.859592 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859585 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-oauth-serving-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:11:08.859592 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859578 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-config" (OuterVolumeSpecName: "console-config") pod "5604af00-83fa-4edc-b258-ae6cc3f8bbc9" (UID: "5604af00-83fa-4edc-b258-ae6cc3f8bbc9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:11:08.859745 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.859594 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-trusted-ca-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:11:08.861702 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.861672 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5604af00-83fa-4edc-b258-ae6cc3f8bbc9" (UID: "5604af00-83fa-4edc-b258-ae6cc3f8bbc9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:11:08.861702 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.861686 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-kube-api-access-8jwrb" (OuterVolumeSpecName: "kube-api-access-8jwrb") pod "5604af00-83fa-4edc-b258-ae6cc3f8bbc9" (UID: "5604af00-83fa-4edc-b258-ae6cc3f8bbc9"). InnerVolumeSpecName "kube-api-access-8jwrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:11:08.861850 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.861699 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5604af00-83fa-4edc-b258-ae6cc3f8bbc9" (UID: "5604af00-83fa-4edc-b258-ae6cc3f8bbc9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:11:08.960783 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.960745 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-serving-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:11:08.960783 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.960773 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-oauth-config\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:11:08.960783 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.960782 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-console-config\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:11:08.960783 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:08.960791 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jwrb\" (UniqueName: \"kubernetes.io/projected/5604af00-83fa-4edc-b258-ae6cc3f8bbc9-kube-api-access-8jwrb\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:11:09.529527 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:09.529492 2556 generic.go:356] "Generic (PLEG): container finished" podID="5604af00-83fa-4edc-b258-ae6cc3f8bbc9" containerID="544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c" exitCode=0 Jul 10 19:11:09.530001 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:09.529546 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5785f94766-d5x62" Jul 10 19:11:09.530001 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:09.529589 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5785f94766-d5x62" event={"ID":"5604af00-83fa-4edc-b258-ae6cc3f8bbc9","Type":"ContainerDied","Data":"544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c"} Jul 10 19:11:09.530001 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:09.529649 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5785f94766-d5x62" event={"ID":"5604af00-83fa-4edc-b258-ae6cc3f8bbc9","Type":"ContainerDied","Data":"9111d5cc7c0f03d12ae680641cb67222eb3786d772f731f50a2cca2b04769b79"} Jul 10 19:11:09.530001 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:09.529674 2556 scope.go:117] "RemoveContainer" containerID="544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c" Jul 10 19:11:09.538628 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:09.538609 2556 scope.go:117] "RemoveContainer" containerID="544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c" Jul 10 19:11:09.538930 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:11:09.538913 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c\": container with ID starting with 544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c not found: ID does not exist" containerID="544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c" Jul 10 19:11:09.538986 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:09.538937 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c"} err="failed to get container status \"544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c\": rpc error: code = NotFound desc = could not find container \"544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c\": container with ID starting with 544b80b58b5200fbeeb9924166b50aa398149519d350b6a157bf85ad90b04a7c not found: ID does not exist" Jul 10 19:11:09.550728 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:09.550697 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5785f94766-d5x62"] Jul 10 19:11:09.557297 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:09.557271 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5785f94766-d5x62"] Jul 10 19:11:10.516382 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:10.516346 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5604af00-83fa-4edc-b258-ae6cc3f8bbc9" path="/var/lib/kubelet/pods/5604af00-83fa-4edc-b258-ae6cc3f8bbc9/volumes" Jul 10 19:11:48.117059 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.116978 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5775766f6c-5hgxn"] Jul 10 19:11:48.117490 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.117412 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5604af00-83fa-4edc-b258-ae6cc3f8bbc9" containerName="console" Jul 10 19:11:48.117490 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.117427 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5604af00-83fa-4edc-b258-ae6cc3f8bbc9" containerName="console" Jul 10 19:11:48.117490 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.117484 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5604af00-83fa-4edc-b258-ae6cc3f8bbc9" containerName="console" Jul 10 19:11:48.120751 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.120727 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.134308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.134271 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5775766f6c-5hgxn"] Jul 10 19:11:48.282254 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.282213 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87ljf\" (UniqueName: \"kubernetes.io/projected/2bd56573-9764-47c7-92ce-09db536d1e9a-kube-api-access-87ljf\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.282436 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.282266 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-serving-cert\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.282436 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.282292 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-trusted-ca-bundle\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.282436 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.282318 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-service-ca\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.282436 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.282348 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-oauth-serving-cert\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.282581 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.282426 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-console-config\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.282581 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.282473 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-oauth-config\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.383954 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.383827 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ljf\" (UniqueName: \"kubernetes.io/projected/2bd56573-9764-47c7-92ce-09db536d1e9a-kube-api-access-87ljf\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.383954 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.383937 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-serving-cert\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.383954 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.383963 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-trusted-ca-bundle\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.384198 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.383982 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-service-ca\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.384198 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.384006 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-oauth-serving-cert\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.384198 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.384035 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-console-config\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.384198 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.384068 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-oauth-config\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.384925 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.384896 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-console-config\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.385036 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.384930 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-service-ca\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.385036 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.385007 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-trusted-ca-bundle\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.385108 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.385048 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-oauth-serving-cert\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.386580 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.386554 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-oauth-config\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.386672 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.386655 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-serving-cert\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.393389 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.393368 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87ljf\" (UniqueName: \"kubernetes.io/projected/2bd56573-9764-47c7-92ce-09db536d1e9a-kube-api-access-87ljf\") pod \"console-5775766f6c-5hgxn\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.431412 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.431369 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:48.571543 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.571516 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5775766f6c-5hgxn"] Jul 10 19:11:48.573961 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:11:48.573919 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd56573_9764_47c7_92ce_09db536d1e9a.slice/crio-508cfdddb897376628c6916de181da36f183015dc9be1019867d92c4e967538d WatchSource:0}: Error finding container 508cfdddb897376628c6916de181da36f183015dc9be1019867d92c4e967538d: Status 404 returned error can't find the container with id 508cfdddb897376628c6916de181da36f183015dc9be1019867d92c4e967538d Jul 10 19:11:48.679626 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.679537 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5775766f6c-5hgxn" event={"ID":"2bd56573-9764-47c7-92ce-09db536d1e9a","Type":"ContainerStarted","Data":"10fe93426fd7ca0c94af4a3a37f19656749ab0b7ae2b005200a3787b6bafbbae"} Jul 10 19:11:48.679626 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.679590 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5775766f6c-5hgxn" event={"ID":"2bd56573-9764-47c7-92ce-09db536d1e9a","Type":"ContainerStarted","Data":"508cfdddb897376628c6916de181da36f183015dc9be1019867d92c4e967538d"} Jul 10 19:11:48.699777 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:48.699722 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5775766f6c-5hgxn" podStartSLOduration=0.699702101 podStartE2EDuration="699.702101ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:11:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:11:48.698506026 +0000 UTC m=+260.714964396" watchObservedRunningTime="2026-07-10 19:11:48.699702101 +0000 UTC m=+260.716160472" Jul 10 19:11:58.432144 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:58.432106 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:58.432678 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:58.432160 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:58.436689 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:58.436662 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:58.726147 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:58.726057 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:11:58.778743 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:11:58.778710 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f66cc496f-dh5j4"] Jul 10 19:12:23.809265 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:23.809212 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f66cc496f-dh5j4" podUID="da552e32-fe26-456e-92c8-712e87a7314c" containerName="console" containerID="cri-o://5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37" gracePeriod=15 Jul 10 19:12:24.058536 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.058503 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:12:24.173105 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.172998 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-service-ca\") pod \"da552e32-fe26-456e-92c8-712e87a7314c\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " Jul 10 19:12:24.173105 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173072 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-oauth-config\") pod \"da552e32-fe26-456e-92c8-712e87a7314c\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " Jul 10 19:12:24.173105 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173092 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-serving-cert\") pod \"da552e32-fe26-456e-92c8-712e87a7314c\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " Jul 10 19:12:24.173392 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173125 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-console-config\") pod \"da552e32-fe26-456e-92c8-712e87a7314c\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " Jul 10 19:12:24.173392 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173163 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wcwc\" (UniqueName: \"kubernetes.io/projected/da552e32-fe26-456e-92c8-712e87a7314c-kube-api-access-6wcwc\") pod \"da552e32-fe26-456e-92c8-712e87a7314c\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " Jul 10 19:12:24.173392 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173209 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-trusted-ca-bundle\") pod \"da552e32-fe26-456e-92c8-712e87a7314c\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " Jul 10 19:12:24.173392 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173230 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-oauth-serving-cert\") pod \"da552e32-fe26-456e-92c8-712e87a7314c\" (UID: \"da552e32-fe26-456e-92c8-712e87a7314c\") " Jul 10 19:12:24.173587 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173457 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-service-ca" (OuterVolumeSpecName: "service-ca") pod "da552e32-fe26-456e-92c8-712e87a7314c" (UID: "da552e32-fe26-456e-92c8-712e87a7314c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:12:24.173645 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173578 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-console-config" (OuterVolumeSpecName: "console-config") pod "da552e32-fe26-456e-92c8-712e87a7314c" (UID: "da552e32-fe26-456e-92c8-712e87a7314c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:12:24.173695 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173640 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "da552e32-fe26-456e-92c8-712e87a7314c" (UID: "da552e32-fe26-456e-92c8-712e87a7314c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:12:24.174013 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.173981 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "da552e32-fe26-456e-92c8-712e87a7314c" (UID: "da552e32-fe26-456e-92c8-712e87a7314c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:12:24.175573 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.175547 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da552e32-fe26-456e-92c8-712e87a7314c-kube-api-access-6wcwc" (OuterVolumeSpecName: "kube-api-access-6wcwc") pod "da552e32-fe26-456e-92c8-712e87a7314c" (UID: "da552e32-fe26-456e-92c8-712e87a7314c"). InnerVolumeSpecName "kube-api-access-6wcwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:12:24.175673 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.175574 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "da552e32-fe26-456e-92c8-712e87a7314c" (UID: "da552e32-fe26-456e-92c8-712e87a7314c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:12:24.175673 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.175629 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "da552e32-fe26-456e-92c8-712e87a7314c" (UID: "da552e32-fe26-456e-92c8-712e87a7314c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:12:24.273920 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.273854 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-oauth-config\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:24.273920 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.273913 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da552e32-fe26-456e-92c8-712e87a7314c-console-serving-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:24.273920 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.273925 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-console-config\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:24.274155 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.273934 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wcwc\" (UniqueName: \"kubernetes.io/projected/da552e32-fe26-456e-92c8-712e87a7314c-kube-api-access-6wcwc\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:24.274155 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.273945 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-trusted-ca-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:24.274155 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.273953 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-oauth-serving-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:24.274155 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.273962 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da552e32-fe26-456e-92c8-712e87a7314c-service-ca\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:24.821342 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.821253 2556 generic.go:356] "Generic (PLEG): container finished" podID="da552e32-fe26-456e-92c8-712e87a7314c" containerID="5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37" exitCode=0 Jul 10 19:12:24.821342 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.821298 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f66cc496f-dh5j4" Jul 10 19:12:24.821829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.821342 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f66cc496f-dh5j4" event={"ID":"da552e32-fe26-456e-92c8-712e87a7314c","Type":"ContainerDied","Data":"5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37"} Jul 10 19:12:24.821829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.821404 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f66cc496f-dh5j4" event={"ID":"da552e32-fe26-456e-92c8-712e87a7314c","Type":"ContainerDied","Data":"62a8b11bd78dc09de0a6d1a5193a3b4d9d7e10cc48e30d94c84f1f98d2967b5e"} Jul 10 19:12:24.821829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.821434 2556 scope.go:117] "RemoveContainer" containerID="5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37" Jul 10 19:12:24.830226 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.830206 2556 scope.go:117] "RemoveContainer" containerID="5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37" Jul 10 19:12:24.830551 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:12:24.830531 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37\": container with ID starting with 5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37 not found: ID does not exist" containerID="5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37" Jul 10 19:12:24.830590 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.830562 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37"} err="failed to get container status \"5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37\": rpc error: code = NotFound desc = could not find container \"5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37\": container with ID starting with 5953180a306c1be9da4d7e34fc4ff630cecc1acbbf5f48c9d7844724c2d3ae37 not found: ID does not exist" Jul 10 19:12:24.877082 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.877047 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f66cc496f-dh5j4"] Jul 10 19:12:24.893928 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:24.893900 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f66cc496f-dh5j4"] Jul 10 19:12:26.516497 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:26.516465 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da552e32-fe26-456e-92c8-712e87a7314c" path="/var/lib/kubelet/pods/da552e32-fe26-456e-92c8-712e87a7314c/volumes" Jul 10 19:12:28.374012 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:28.373957 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:12:28.378162 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:28.378117 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:12:28.381932 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:28.381901 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:12:28.386233 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:28.386208 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:12:28.386401 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:28.386265 2556 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 19:12:38.338785 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.338746 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9"] Jul 10 19:12:38.341199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.339097 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da552e32-fe26-456e-92c8-712e87a7314c" containerName="console" Jul 10 19:12:38.341199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.339111 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="da552e32-fe26-456e-92c8-712e87a7314c" containerName="console" Jul 10 19:12:38.341199 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.339161 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="da552e32-fe26-456e-92c8-712e87a7314c" containerName="console" Jul 10 19:12:38.342278 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.342256 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.345144 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.345116 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 19:12:38.345975 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.345957 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 19:12:38.346062 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.345996 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gqrrr\"" Jul 10 19:12:38.353576 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.353549 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9"] Jul 10 19:12:38.499793 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.499761 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.500016 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.499821 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.500016 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.499862 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtwsd\" (UniqueName: \"kubernetes.io/projected/6a173b41-7782-4f34-ae70-40cea617aaa2-kube-api-access-rtwsd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.600543 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.600453 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.600714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.600691 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.600774 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.600732 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtwsd\" (UniqueName: \"kubernetes.io/projected/6a173b41-7782-4f34-ae70-40cea617aaa2-kube-api-access-rtwsd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.600928 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.600905 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.601074 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.601053 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.609240 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.609212 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtwsd\" (UniqueName: \"kubernetes.io/projected/6a173b41-7782-4f34-ae70-40cea617aaa2-kube-api-access-rtwsd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.651938 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.651892 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:38.798003 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.797969 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9"] Jul 10 19:12:38.800619 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:12:38.800591 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a173b41_7782_4f34_ae70_40cea617aaa2.slice/crio-f7f4558ca5c1ec69b370fd78bc83d50e13c79035a590f2898db440953a003952 WatchSource:0}: Error finding container f7f4558ca5c1ec69b370fd78bc83d50e13c79035a590f2898db440953a003952: Status 404 returned error can't find the container with id f7f4558ca5c1ec69b370fd78bc83d50e13c79035a590f2898db440953a003952 Jul 10 19:12:38.802672 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.802652 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:12:38.874996 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:38.874906 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" event={"ID":"6a173b41-7782-4f34-ae70-40cea617aaa2","Type":"ContainerStarted","Data":"f7f4558ca5c1ec69b370fd78bc83d50e13c79035a590f2898db440953a003952"} Jul 10 19:12:44.902801 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:44.902765 2556 generic.go:356] "Generic (PLEG): container finished" podID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerID="86ab2bc00cbbcfcb25aebee89f28f7ea2fabfaa91f6e722489c39314ee16e138" exitCode=0 Jul 10 19:12:44.903211 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:44.902811 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" event={"ID":"6a173b41-7782-4f34-ae70-40cea617aaa2","Type":"ContainerDied","Data":"86ab2bc00cbbcfcb25aebee89f28f7ea2fabfaa91f6e722489c39314ee16e138"} Jul 10 19:12:47.916443 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:47.916404 2556 generic.go:356] "Generic (PLEG): container finished" podID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerID="f8955a7336b68d93a3ab099fd2c909804ffd05360ad15209d57452135181364f" exitCode=0 Jul 10 19:12:47.916922 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:47.916488 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" event={"ID":"6a173b41-7782-4f34-ae70-40cea617aaa2","Type":"ContainerDied","Data":"f8955a7336b68d93a3ab099fd2c909804ffd05360ad15209d57452135181364f"} Jul 10 19:12:55.953369 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:55.953331 2556 generic.go:356] "Generic (PLEG): container finished" podID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerID="c85a5dccf073a940b49b943a4097ae4d8fddaca7f9fbbce006a53ce2ca8beb83" exitCode=0 Jul 10 19:12:55.953811 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:55.953391 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" event={"ID":"6a173b41-7782-4f34-ae70-40cea617aaa2","Type":"ContainerDied","Data":"c85a5dccf073a940b49b943a4097ae4d8fddaca7f9fbbce006a53ce2ca8beb83"} Jul 10 19:12:57.096585 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.096558 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:57.156134 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.156093 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtwsd\" (UniqueName: \"kubernetes.io/projected/6a173b41-7782-4f34-ae70-40cea617aaa2-kube-api-access-rtwsd\") pod \"6a173b41-7782-4f34-ae70-40cea617aaa2\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " Jul 10 19:12:57.156309 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.156145 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-util\") pod \"6a173b41-7782-4f34-ae70-40cea617aaa2\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " Jul 10 19:12:57.156309 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.156215 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-bundle\") pod \"6a173b41-7782-4f34-ae70-40cea617aaa2\" (UID: \"6a173b41-7782-4f34-ae70-40cea617aaa2\") " Jul 10 19:12:57.157009 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.156922 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-bundle" (OuterVolumeSpecName: "bundle") pod "6a173b41-7782-4f34-ae70-40cea617aaa2" (UID: "6a173b41-7782-4f34-ae70-40cea617aaa2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:12:57.158603 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.158572 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a173b41-7782-4f34-ae70-40cea617aaa2-kube-api-access-rtwsd" (OuterVolumeSpecName: "kube-api-access-rtwsd") pod "6a173b41-7782-4f34-ae70-40cea617aaa2" (UID: "6a173b41-7782-4f34-ae70-40cea617aaa2"). InnerVolumeSpecName "kube-api-access-rtwsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:12:57.160367 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.160340 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-util" (OuterVolumeSpecName: "util") pod "6a173b41-7782-4f34-ae70-40cea617aaa2" (UID: "6a173b41-7782-4f34-ae70-40cea617aaa2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:12:57.257061 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.257025 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-util\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:57.257061 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.257057 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a173b41-7782-4f34-ae70-40cea617aaa2-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:57.257061 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.257067 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtwsd\" (UniqueName: \"kubernetes.io/projected/6a173b41-7782-4f34-ae70-40cea617aaa2-kube-api-access-rtwsd\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:12:57.962913 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.962864 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" Jul 10 19:12:57.963088 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.962863 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gvzn9" event={"ID":"6a173b41-7782-4f34-ae70-40cea617aaa2","Type":"ContainerDied","Data":"f7f4558ca5c1ec69b370fd78bc83d50e13c79035a590f2898db440953a003952"} Jul 10 19:12:57.963088 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:12:57.963004 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7f4558ca5c1ec69b370fd78bc83d50e13c79035a590f2898db440953a003952" Jul 10 19:13:01.341273 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.341238 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j"] Jul 10 19:13:01.341637 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.341574 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerName="util" Jul 10 19:13:01.341637 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.341593 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerName="util" Jul 10 19:13:01.341637 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.341622 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerName="pull" Jul 10 19:13:01.341637 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.341630 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerName="pull" Jul 10 19:13:01.341762 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.341649 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerName="extract" Jul 10 19:13:01.341762 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.341659 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerName="extract" Jul 10 19:13:01.341820 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.341772 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a173b41-7782-4f34-ae70-40cea617aaa2" containerName="extract" Jul 10 19:13:01.361967 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.361938 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j"] Jul 10 19:13:01.362122 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.362034 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" Jul 10 19:13:01.365179 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.365150 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 10 19:13:01.365319 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.365150 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-62dbn\"" Jul 10 19:13:01.365319 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.365156 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 10 19:13:01.386829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.386791 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fb138b4-7378-4577-a10b-e15fe248e732-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq86j\" (UID: \"5fb138b4-7378-4577-a10b-e15fe248e732\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" Jul 10 19:13:01.387001 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.386839 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qfmp\" (UniqueName: \"kubernetes.io/projected/5fb138b4-7378-4577-a10b-e15fe248e732-kube-api-access-7qfmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq86j\" (UID: \"5fb138b4-7378-4577-a10b-e15fe248e732\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" Jul 10 19:13:01.488146 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.488095 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fb138b4-7378-4577-a10b-e15fe248e732-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq86j\" (UID: \"5fb138b4-7378-4577-a10b-e15fe248e732\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" Jul 10 19:13:01.488313 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.488156 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qfmp\" (UniqueName: \"kubernetes.io/projected/5fb138b4-7378-4577-a10b-e15fe248e732-kube-api-access-7qfmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq86j\" (UID: \"5fb138b4-7378-4577-a10b-e15fe248e732\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" Jul 10 19:13:01.488591 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.488567 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fb138b4-7378-4577-a10b-e15fe248e732-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq86j\" (UID: \"5fb138b4-7378-4577-a10b-e15fe248e732\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" Jul 10 19:13:01.499535 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.499503 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qfmp\" (UniqueName: \"kubernetes.io/projected/5fb138b4-7378-4577-a10b-e15fe248e732-kube-api-access-7qfmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq86j\" (UID: \"5fb138b4-7378-4577-a10b-e15fe248e732\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" Jul 10 19:13:01.671895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.671765 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" Jul 10 19:13:01.848150 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.848061 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j"] Jul 10 19:13:01.850686 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:13:01.850657 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb138b4_7378_4577_a10b_e15fe248e732.slice/crio-9963f677beba536ee374c56dacd54bab97c85f92f64be93215097f7b361f5240 WatchSource:0}: Error finding container 9963f677beba536ee374c56dacd54bab97c85f92f64be93215097f7b361f5240: Status 404 returned error can't find the container with id 9963f677beba536ee374c56dacd54bab97c85f92f64be93215097f7b361f5240 Jul 10 19:13:01.980294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:01.980209 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" event={"ID":"5fb138b4-7378-4577-a10b-e15fe248e732","Type":"ContainerStarted","Data":"9963f677beba536ee374c56dacd54bab97c85f92f64be93215097f7b361f5240"} Jul 10 19:13:04.995615 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:04.995564 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" event={"ID":"5fb138b4-7378-4577-a10b-e15fe248e732","Type":"ContainerStarted","Data":"8d84522aa2a7df1228a0bdb611515fce8696a161fcd388cd5b12b5e2633b81e5"} Jul 10 19:13:05.019336 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.019242 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq86j" podStartSLOduration=1.925733352 podStartE2EDuration="4.01921884s" totalImagesPullingTime="2.093485488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:13:05.01699869 +0000 UTC m=+337.033457088" watchObservedRunningTime="2026-07-10 19:13:05.01921884 +0000 UTC m=+337.035677211" Jul 10 19:13:05.229676 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.229639 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cnlwx"] Jul 10 19:13:05.250948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.250854 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cnlwx"] Jul 10 19:13:05.251090 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.250997 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:05.254079 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.254050 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-l9j68\"" Jul 10 19:13:05.254255 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.254051 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 19:13:05.254255 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.254050 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 19:13:05.317935 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.317887 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff0017fc-1aa6-49dc-931c-d95e42adedd0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cnlwx\" (UID: \"ff0017fc-1aa6-49dc-931c-d95e42adedd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:05.318127 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.317946 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jrcf\" (UniqueName: \"kubernetes.io/projected/ff0017fc-1aa6-49dc-931c-d95e42adedd0-kube-api-access-8jrcf\") pod \"cert-manager-webhook-6486ffd997-cnlwx\" (UID: \"ff0017fc-1aa6-49dc-931c-d95e42adedd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:05.418814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.418777 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff0017fc-1aa6-49dc-931c-d95e42adedd0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cnlwx\" (UID: \"ff0017fc-1aa6-49dc-931c-d95e42adedd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:05.418814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.418822 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jrcf\" (UniqueName: \"kubernetes.io/projected/ff0017fc-1aa6-49dc-931c-d95e42adedd0-kube-api-access-8jrcf\") pod \"cert-manager-webhook-6486ffd997-cnlwx\" (UID: \"ff0017fc-1aa6-49dc-931c-d95e42adedd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:05.437040 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.437000 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff0017fc-1aa6-49dc-931c-d95e42adedd0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cnlwx\" (UID: \"ff0017fc-1aa6-49dc-931c-d95e42adedd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:05.441428 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.441392 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jrcf\" (UniqueName: \"kubernetes.io/projected/ff0017fc-1aa6-49dc-931c-d95e42adedd0-kube-api-access-8jrcf\") pod \"cert-manager-webhook-6486ffd997-cnlwx\" (UID: \"ff0017fc-1aa6-49dc-931c-d95e42adedd0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:05.569611 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.569513 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:05.730593 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.730555 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll"] Jul 10 19:13:05.743314 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:13:05.743288 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff0017fc_1aa6_49dc_931c_d95e42adedd0.slice/crio-7197dac603bc34eb4f3c54679808c593ffb0467ec606a429d04915a6bb2a852c WatchSource:0}: Error finding container 7197dac603bc34eb4f3c54679808c593ffb0467ec606a429d04915a6bb2a852c: Status 404 returned error can't find the container with id 7197dac603bc34eb4f3c54679808c593ffb0467ec606a429d04915a6bb2a852c Jul 10 19:13:05.760617 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.760586 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cnlwx"] Jul 10 19:13:05.760765 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.760626 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll"] Jul 10 19:13:05.760765 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.760706 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:05.764241 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.764158 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gqrrr\"" Jul 10 19:13:05.764241 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.764172 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 19:13:05.765428 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.765396 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 19:13:05.821988 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.821862 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:05.821988 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.821949 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:05.822173 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.821999 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq2fp\" (UniqueName: \"kubernetes.io/projected/e10d09f6-7822-48b3-87e1-82fbfc49cac0-kube-api-access-tq2fp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:05.923154 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.923112 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq2fp\" (UniqueName: \"kubernetes.io/projected/e10d09f6-7822-48b3-87e1-82fbfc49cac0-kube-api-access-tq2fp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:05.923364 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.923173 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:05.923364 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.923197 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:05.923547 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.923529 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:05.923671 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.923649 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:05.932502 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:05.932466 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq2fp\" (UniqueName: \"kubernetes.io/projected/e10d09f6-7822-48b3-87e1-82fbfc49cac0-kube-api-access-tq2fp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:06.001096 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:06.001054 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" event={"ID":"ff0017fc-1aa6-49dc-931c-d95e42adedd0","Type":"ContainerStarted","Data":"7197dac603bc34eb4f3c54679808c593ffb0467ec606a429d04915a6bb2a852c"} Jul 10 19:13:06.071728 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:06.071687 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:06.233171 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:06.233137 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll"] Jul 10 19:13:06.236592 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:13:06.236562 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10d09f6_7822_48b3_87e1_82fbfc49cac0.slice/crio-cb9c11147da4be0de5c30511828f33ec57c9e2403d64c7c520cd262ffb05539e WatchSource:0}: Error finding container cb9c11147da4be0de5c30511828f33ec57c9e2403d64c7c520cd262ffb05539e: Status 404 returned error can't find the container with id cb9c11147da4be0de5c30511828f33ec57c9e2403d64c7c520cd262ffb05539e Jul 10 19:13:07.008109 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:07.008072 2556 generic.go:356] "Generic (PLEG): container finished" podID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerID="8ccf22f2d637529e6416744ad89c6d1bcf224f9e90bdcd431864d5a810c16e77" exitCode=0 Jul 10 19:13:07.008568 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:07.008119 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" event={"ID":"e10d09f6-7822-48b3-87e1-82fbfc49cac0","Type":"ContainerDied","Data":"8ccf22f2d637529e6416744ad89c6d1bcf224f9e90bdcd431864d5a810c16e77"} Jul 10 19:13:07.008568 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:07.008165 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" event={"ID":"e10d09f6-7822-48b3-87e1-82fbfc49cac0","Type":"ContainerStarted","Data":"cb9c11147da4be0de5c30511828f33ec57c9e2403d64c7c520cd262ffb05539e"} Jul 10 19:13:08.875142 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:08.875085 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j"] Jul 10 19:13:08.877937 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:08.877896 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" Jul 10 19:13:08.880916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:08.880823 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-55zfp\"" Jul 10 19:13:08.888354 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:08.888323 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j"] Jul 10 19:13:08.952130 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:08.952038 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0022c5f5-15c1-4487-a8be-cac37e3691e4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-v7h8j\" (UID: \"0022c5f5-15c1-4487-a8be-cac37e3691e4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" Jul 10 19:13:08.952311 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:08.952186 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8t2w\" (UniqueName: \"kubernetes.io/projected/0022c5f5-15c1-4487-a8be-cac37e3691e4-kube-api-access-p8t2w\") pod \"cert-manager-cainjector-559cdd8c87-v7h8j\" (UID: \"0022c5f5-15c1-4487-a8be-cac37e3691e4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" Jul 10 19:13:09.027744 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:09.027692 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" event={"ID":"ff0017fc-1aa6-49dc-931c-d95e42adedd0","Type":"ContainerStarted","Data":"bba2d83ff5c104e14c53b4df8da23ab48723ff2fe6a5979f10e4e62fae4b303a"} Jul 10 19:13:09.027961 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:09.027759 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:09.048856 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:09.048792 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" podStartSLOduration=1.142962169 podStartE2EDuration="4.048772247s" totalImagesPullingTime="2.905810078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:13:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:13:09.047137726 +0000 UTC m=+341.063596096" watchObservedRunningTime="2026-07-10 19:13:09.048772247 +0000 UTC m=+341.065230618" Jul 10 19:13:09.053550 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:09.053512 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8t2w\" (UniqueName: \"kubernetes.io/projected/0022c5f5-15c1-4487-a8be-cac37e3691e4-kube-api-access-p8t2w\") pod \"cert-manager-cainjector-559cdd8c87-v7h8j\" (UID: \"0022c5f5-15c1-4487-a8be-cac37e3691e4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" Jul 10 19:13:09.053750 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:09.053584 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0022c5f5-15c1-4487-a8be-cac37e3691e4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-v7h8j\" (UID: \"0022c5f5-15c1-4487-a8be-cac37e3691e4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" Jul 10 19:13:09.062946 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:09.062916 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0022c5f5-15c1-4487-a8be-cac37e3691e4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-v7h8j\" (UID: \"0022c5f5-15c1-4487-a8be-cac37e3691e4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" Jul 10 19:13:09.063219 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:09.063192 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8t2w\" (UniqueName: \"kubernetes.io/projected/0022c5f5-15c1-4487-a8be-cac37e3691e4-kube-api-access-p8t2w\") pod \"cert-manager-cainjector-559cdd8c87-v7h8j\" (UID: \"0022c5f5-15c1-4487-a8be-cac37e3691e4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" Jul 10 19:13:09.189961 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:09.189918 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" Jul 10 19:13:09.347791 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:09.347754 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j"] Jul 10 19:13:09.954724 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:13:09.954687 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0022c5f5_15c1_4487_a8be_cac37e3691e4.slice/crio-fc977af3b82c31083928fcad4f4339a514a5e830726bfe58bc83e9e4aaebbbbc WatchSource:0}: Error finding container fc977af3b82c31083928fcad4f4339a514a5e830726bfe58bc83e9e4aaebbbbc: Status 404 returned error can't find the container with id fc977af3b82c31083928fcad4f4339a514a5e830726bfe58bc83e9e4aaebbbbc Jul 10 19:13:10.034491 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:10.034455 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" event={"ID":"0022c5f5-15c1-4487-a8be-cac37e3691e4","Type":"ContainerStarted","Data":"fc977af3b82c31083928fcad4f4339a514a5e830726bfe58bc83e9e4aaebbbbc"} Jul 10 19:13:11.040464 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:11.040425 2556 generic.go:356] "Generic (PLEG): container finished" podID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerID="5b3ac3ef84ae1f7dd19b6460e1d89e2c6f72d2c5dd23867fa89f16fa504ca634" exitCode=0 Jul 10 19:13:11.040913 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:11.040516 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" event={"ID":"e10d09f6-7822-48b3-87e1-82fbfc49cac0","Type":"ContainerDied","Data":"5b3ac3ef84ae1f7dd19b6460e1d89e2c6f72d2c5dd23867fa89f16fa504ca634"} Jul 10 19:13:11.042223 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:11.042184 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" event={"ID":"0022c5f5-15c1-4487-a8be-cac37e3691e4","Type":"ContainerStarted","Data":"6ceb19bf8c1c50f3ad4764509431e523cdc615d9cb24db21df8abf11d452a4d0"} Jul 10 19:13:11.083486 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:11.083432 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-v7h8j" podStartSLOduration=3.083416547 podStartE2EDuration="3.083416547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:13:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:13:11.081753007 +0000 UTC m=+343.098211382" watchObservedRunningTime="2026-07-10 19:13:11.083416547 +0000 UTC m=+343.099874918" Jul 10 19:13:12.049210 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:12.049169 2556 generic.go:356] "Generic (PLEG): container finished" podID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerID="bfeb491e0a8fd18491c704f595dd0d46f72b7f6be98ca9a02c72d5a4197139c3" exitCode=0 Jul 10 19:13:12.049599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:12.049250 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" event={"ID":"e10d09f6-7822-48b3-87e1-82fbfc49cac0","Type":"ContainerDied","Data":"bfeb491e0a8fd18491c704f595dd0d46f72b7f6be98ca9a02c72d5a4197139c3"} Jul 10 19:13:13.195474 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.195448 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:13.289270 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.289228 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-util\") pod \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " Jul 10 19:13:13.289440 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.289318 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-bundle\") pod \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " Jul 10 19:13:13.289440 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.289360 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq2fp\" (UniqueName: \"kubernetes.io/projected/e10d09f6-7822-48b3-87e1-82fbfc49cac0-kube-api-access-tq2fp\") pod \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\" (UID: \"e10d09f6-7822-48b3-87e1-82fbfc49cac0\") " Jul 10 19:13:13.289666 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.289644 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-bundle" (OuterVolumeSpecName: "bundle") pod "e10d09f6-7822-48b3-87e1-82fbfc49cac0" (UID: "e10d09f6-7822-48b3-87e1-82fbfc49cac0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:13:13.291601 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.291565 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e10d09f6-7822-48b3-87e1-82fbfc49cac0-kube-api-access-tq2fp" (OuterVolumeSpecName: "kube-api-access-tq2fp") pod "e10d09f6-7822-48b3-87e1-82fbfc49cac0" (UID: "e10d09f6-7822-48b3-87e1-82fbfc49cac0"). InnerVolumeSpecName "kube-api-access-tq2fp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:13:13.294280 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.294247 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-util" (OuterVolumeSpecName: "util") pod "e10d09f6-7822-48b3-87e1-82fbfc49cac0" (UID: "e10d09f6-7822-48b3-87e1-82fbfc49cac0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:13:13.390175 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.390088 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-util\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:13:13.390175 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.390120 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e10d09f6-7822-48b3-87e1-82fbfc49cac0-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:13:13.390175 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:13.390129 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq2fp\" (UniqueName: \"kubernetes.io/projected/e10d09f6-7822-48b3-87e1-82fbfc49cac0-kube-api-access-tq2fp\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:13:14.062551 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:14.062523 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" Jul 10 19:13:14.062551 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:14.062533 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxsqll" event={"ID":"e10d09f6-7822-48b3-87e1-82fbfc49cac0","Type":"ContainerDied","Data":"cb9c11147da4be0de5c30511828f33ec57c9e2403d64c7c520cd262ffb05539e"} Jul 10 19:13:14.062757 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:14.062576 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb9c11147da4be0de5c30511828f33ec57c9e2403d64c7c520cd262ffb05539e" Jul 10 19:13:15.037097 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:15.037069 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cnlwx" Jul 10 19:13:31.087928 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.087804 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf"] Jul 10 19:13:31.088383 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.088308 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerName="util" Jul 10 19:13:31.088383 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.088328 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerName="util" Jul 10 19:13:31.088383 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.088352 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerName="extract" Jul 10 19:13:31.088383 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.088361 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerName="extract" Jul 10 19:13:31.088383 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.088371 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerName="pull" Jul 10 19:13:31.088383 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.088379 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerName="pull" Jul 10 19:13:31.088676 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.088452 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e10d09f6-7822-48b3-87e1-82fbfc49cac0" containerName="extract" Jul 10 19:13:31.090764 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.090743 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.093265 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.093215 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 19:13:31.094181 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.094163 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 19:13:31.094297 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.094184 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gqrrr\"" Jul 10 19:13:31.100063 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.100040 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf"] Jul 10 19:13:31.246864 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.246827 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.247043 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.247004 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnzxp\" (UniqueName: \"kubernetes.io/projected/78a52676-a10f-4a8b-8626-2e7f92de09f0-kube-api-access-bnzxp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.247079 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.247059 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.348129 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.348021 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.348129 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.348108 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnzxp\" (UniqueName: \"kubernetes.io/projected/78a52676-a10f-4a8b-8626-2e7f92de09f0-kube-api-access-bnzxp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.348129 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.348140 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.348429 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.348409 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.348518 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.348438 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.357111 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.357079 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnzxp\" (UniqueName: \"kubernetes.io/projected/78a52676-a10f-4a8b-8626-2e7f92de09f0-kube-api-access-bnzxp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.401320 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.401273 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:31.542436 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:31.542340 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf"] Jul 10 19:13:31.544688 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:13:31.544660 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a52676_a10f_4a8b_8626_2e7f92de09f0.slice/crio-96c87a2e8969afb362623c05c276e3ac094b3db4a7bdc8bfb4712c38303c850a WatchSource:0}: Error finding container 96c87a2e8969afb362623c05c276e3ac094b3db4a7bdc8bfb4712c38303c850a: Status 404 returned error can't find the container with id 96c87a2e8969afb362623c05c276e3ac094b3db4a7bdc8bfb4712c38303c850a Jul 10 19:13:32.143743 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:32.143711 2556 generic.go:356] "Generic (PLEG): container finished" podID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerID="5ffe6a5dfcde30b45c6170e61c2c89ae43ee982891c98468b3cae494f7e23745" exitCode=0 Jul 10 19:13:32.144149 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:32.143749 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" event={"ID":"78a52676-a10f-4a8b-8626-2e7f92de09f0","Type":"ContainerDied","Data":"5ffe6a5dfcde30b45c6170e61c2c89ae43ee982891c98468b3cae494f7e23745"} Jul 10 19:13:32.144149 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:32.143784 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" event={"ID":"78a52676-a10f-4a8b-8626-2e7f92de09f0","Type":"ContainerStarted","Data":"96c87a2e8969afb362623c05c276e3ac094b3db4a7bdc8bfb4712c38303c850a"} Jul 10 19:13:33.150122 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:33.150084 2556 generic.go:356] "Generic (PLEG): container finished" podID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerID="888fa38b4d87f44889b7dbd1e6f27e973b3e7886eb0dac9d7d40738026f70c92" exitCode=0 Jul 10 19:13:33.150557 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:33.150166 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" event={"ID":"78a52676-a10f-4a8b-8626-2e7f92de09f0","Type":"ContainerDied","Data":"888fa38b4d87f44889b7dbd1e6f27e973b3e7886eb0dac9d7d40738026f70c92"} Jul 10 19:13:34.157349 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:34.157310 2556 generic.go:356] "Generic (PLEG): container finished" podID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerID="4e918b6f78a9cae7aafeaf5126b2e03557e152f75c4470a282d41508e15d2c69" exitCode=0 Jul 10 19:13:34.157746 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:34.157354 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" event={"ID":"78a52676-a10f-4a8b-8626-2e7f92de09f0","Type":"ContainerDied","Data":"4e918b6f78a9cae7aafeaf5126b2e03557e152f75c4470a282d41508e15d2c69"} Jul 10 19:13:35.296467 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.296434 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:35.384242 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.384203 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-bundle\") pod \"78a52676-a10f-4a8b-8626-2e7f92de09f0\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " Jul 10 19:13:35.384396 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.384252 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnzxp\" (UniqueName: \"kubernetes.io/projected/78a52676-a10f-4a8b-8626-2e7f92de09f0-kube-api-access-bnzxp\") pod \"78a52676-a10f-4a8b-8626-2e7f92de09f0\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " Jul 10 19:13:35.384396 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.384295 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-util\") pod \"78a52676-a10f-4a8b-8626-2e7f92de09f0\" (UID: \"78a52676-a10f-4a8b-8626-2e7f92de09f0\") " Jul 10 19:13:35.385014 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.384952 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-bundle" (OuterVolumeSpecName: "bundle") pod "78a52676-a10f-4a8b-8626-2e7f92de09f0" (UID: "78a52676-a10f-4a8b-8626-2e7f92de09f0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:13:35.386630 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.386605 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78a52676-a10f-4a8b-8626-2e7f92de09f0-kube-api-access-bnzxp" (OuterVolumeSpecName: "kube-api-access-bnzxp") pod "78a52676-a10f-4a8b-8626-2e7f92de09f0" (UID: "78a52676-a10f-4a8b-8626-2e7f92de09f0"). InnerVolumeSpecName "kube-api-access-bnzxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:13:35.392988 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.392954 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-util" (OuterVolumeSpecName: "util") pod "78a52676-a10f-4a8b-8626-2e7f92de09f0" (UID: "78a52676-a10f-4a8b-8626-2e7f92de09f0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:13:35.485180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.485104 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:13:35.485180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.485130 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnzxp\" (UniqueName: \"kubernetes.io/projected/78a52676-a10f-4a8b-8626-2e7f92de09f0-kube-api-access-bnzxp\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:13:35.485180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:35.485140 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78a52676-a10f-4a8b-8626-2e7f92de09f0-util\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:13:36.167921 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:36.167884 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" event={"ID":"78a52676-a10f-4a8b-8626-2e7f92de09f0","Type":"ContainerDied","Data":"96c87a2e8969afb362623c05c276e3ac094b3db4a7bdc8bfb4712c38303c850a"} Jul 10 19:13:36.168167 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:36.167939 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96c87a2e8969afb362623c05c276e3ac094b3db4a7bdc8bfb4712c38303c850a" Jul 10 19:13:36.168167 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:36.167894 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t7nf" Jul 10 19:13:44.321914 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.321857 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp"] Jul 10 19:13:44.322314 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.322200 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerName="pull" Jul 10 19:13:44.322314 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.322211 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerName="pull" Jul 10 19:13:44.322314 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.322221 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerName="util" Jul 10 19:13:44.322314 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.322227 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerName="util" Jul 10 19:13:44.322314 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.322232 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerName="extract" Jul 10 19:13:44.322314 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.322238 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerName="extract" Jul 10 19:13:44.322314 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.322302 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="78a52676-a10f-4a8b-8626-2e7f92de09f0" containerName="extract" Jul 10 19:13:44.326518 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.326497 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.332077 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.332049 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 19:13:44.332703 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.332674 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 19:13:44.332842 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.332699 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 19:13:44.332842 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.332731 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4cjwg\"" Jul 10 19:13:44.333388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.333119 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 19:13:44.373339 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.373303 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp"] Jul 10 19:13:44.461460 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.461420 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gnq9\" (UniqueName: \"kubernetes.io/projected/b5b67215-f589-4689-b233-2179378b9690-kube-api-access-7gnq9\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.461460 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.461464 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5b67215-f589-4689-b233-2179378b9690-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.461681 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.461485 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5b67215-f589-4689-b233-2179378b9690-tmp\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.461681 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.461510 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5b67215-f589-4689-b233-2179378b9690-manifests\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.461681 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.461625 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5b67215-f589-4689-b233-2179378b9690-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.562686 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.562643 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gnq9\" (UniqueName: \"kubernetes.io/projected/b5b67215-f589-4689-b233-2179378b9690-kube-api-access-7gnq9\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.562686 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.562686 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5b67215-f589-4689-b233-2179378b9690-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.563008 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.562704 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5b67215-f589-4689-b233-2179378b9690-tmp\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.563008 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.562834 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5b67215-f589-4689-b233-2179378b9690-manifests\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.563008 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.562974 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5b67215-f589-4689-b233-2179378b9690-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.563230 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.563203 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5b67215-f589-4689-b233-2179378b9690-manifests\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.565225 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.565202 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5b67215-f589-4689-b233-2179378b9690-tmp\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.565553 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.565533 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5b67215-f589-4689-b233-2179378b9690-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.565596 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.565533 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5b67215-f589-4689-b233-2179378b9690-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.573977 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.573907 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gnq9\" (UniqueName: \"kubernetes.io/projected/b5b67215-f589-4689-b233-2179378b9690-kube-api-access-7gnq9\") pod \"opendatahub-operator-controller-manager-64d686dc7f-vdpbp\" (UID: \"b5b67215-f589-4689-b233-2179378b9690\") " pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.637320 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.637282 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:44.783921 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:44.783859 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp"] Jul 10 19:13:44.786345 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:13:44.786313 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b67215_f589_4689_b233_2179378b9690.slice/crio-0b4a2496c2b1a1a0105fe4441c074c155627fa48d80a07e7866ee3716e31a764 WatchSource:0}: Error finding container 0b4a2496c2b1a1a0105fe4441c074c155627fa48d80a07e7866ee3716e31a764: Status 404 returned error can't find the container with id 0b4a2496c2b1a1a0105fe4441c074c155627fa48d80a07e7866ee3716e31a764 Jul 10 19:13:45.216393 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:45.216354 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" event={"ID":"b5b67215-f589-4689-b233-2179378b9690","Type":"ContainerStarted","Data":"0b4a2496c2b1a1a0105fe4441c074c155627fa48d80a07e7866ee3716e31a764"} Jul 10 19:13:48.233045 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:48.232953 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" event={"ID":"b5b67215-f589-4689-b233-2179378b9690","Type":"ContainerStarted","Data":"ef3f47e555ecd2c31bdb0282f85f5016b589464de227d0a292d703a9648df855"} Jul 10 19:13:49.239118 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:49.239083 2556 generic.go:356] "Generic (PLEG): container finished" podID="b5b67215-f589-4689-b233-2179378b9690" containerID="ef3f47e555ecd2c31bdb0282f85f5016b589464de227d0a292d703a9648df855" exitCode=0 Jul 10 19:13:49.239503 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:49.239131 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" event={"ID":"b5b67215-f589-4689-b233-2179378b9690","Type":"ContainerDied","Data":"ef3f47e555ecd2c31bdb0282f85f5016b589464de227d0a292d703a9648df855"} Jul 10 19:13:50.246353 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:50.246298 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" event={"ID":"b5b67215-f589-4689-b233-2179378b9690","Type":"ContainerStarted","Data":"febaf59690271349d36069944bc5a4929743683594fc624cefb8b3d8af364aea"} Jul 10 19:13:50.246824 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:50.246388 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:13:50.275007 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:13:50.274952 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" podStartSLOduration=1.877088697 podStartE2EDuration="6.274933084s" totalImagesPullingTime="3.344781394s" totalInitContainerRuntime="1.053062993s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:13:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:13:50.272177519 +0000 UTC m=+382.288635891" watchObservedRunningTime="2026-07-10 19:13:50.274933084 +0000 UTC m=+382.291391456" Jul 10 19:14:01.254334 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:01.254302 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64d686dc7f-vdpbp" Jul 10 19:14:04.547895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.547832 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-967644b46-jmc4g"] Jul 10 19:14:04.556576 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.556541 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:04.558823 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.558796 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 19:14:04.559363 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.559341 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 19:14:04.575967 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.575936 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-967644b46-jmc4g"] Jul 10 19:14:04.614832 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.614787 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9ltk\" (UniqueName: \"kubernetes.io/projected/901caad2-0055-499e-a5d5-77e23ed88deb-kube-api-access-c9ltk\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:04.615047 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.614853 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/901caad2-0055-499e-a5d5-77e23ed88deb-tmp\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:04.615047 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.614962 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/901caad2-0055-499e-a5d5-77e23ed88deb-tls-certs\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:04.716056 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.716013 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/901caad2-0055-499e-a5d5-77e23ed88deb-tmp\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:04.716220 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.716074 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/901caad2-0055-499e-a5d5-77e23ed88deb-tls-certs\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:04.716220 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.716157 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9ltk\" (UniqueName: \"kubernetes.io/projected/901caad2-0055-499e-a5d5-77e23ed88deb-kube-api-access-c9ltk\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:04.716290 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:04.716243 2556 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 10 19:14:04.716345 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:04.716333 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901caad2-0055-499e-a5d5-77e23ed88deb-tls-certs podName:901caad2-0055-499e-a5d5-77e23ed88deb nodeName:}" failed. No retries permitted until 2026-07-10 19:14:05.216312864 +0000 UTC m=+397.232771216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/901caad2-0055-499e-a5d5-77e23ed88deb-tls-certs") pod "kube-auth-proxy-967644b46-jmc4g" (UID: "901caad2-0055-499e-a5d5-77e23ed88deb") : secret "kube-auth-proxy-tls" not found Jul 10 19:14:04.718753 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.718726 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/901caad2-0055-499e-a5d5-77e23ed88deb-tmp\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:04.752830 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.752791 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9ltk\" (UniqueName: \"kubernetes.io/projected/901caad2-0055-499e-a5d5-77e23ed88deb-kube-api-access-c9ltk\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:04.959527 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.959428 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xbsqz"] Jul 10 19:14:04.968072 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.968031 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:04.978194 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.978119 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 19:14:04.980913 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.980848 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sz6xw\"" Jul 10 19:14:04.981541 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:04.981511 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xbsqz"] Jul 10 19:14:05.018087 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.018048 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5f4c650-01f9-4513-9951-cf1637e933be-cert\") pod \"odh-model-controller-7998c67dc4-xbsqz\" (UID: \"c5f4c650-01f9-4513-9951-cf1637e933be\") " pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:05.018087 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.018091 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g9fl\" (UniqueName: \"kubernetes.io/projected/c5f4c650-01f9-4513-9951-cf1637e933be-kube-api-access-6g9fl\") pod \"odh-model-controller-7998c67dc4-xbsqz\" (UID: \"c5f4c650-01f9-4513-9951-cf1637e933be\") " pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:05.118674 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.118634 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5f4c650-01f9-4513-9951-cf1637e933be-cert\") pod \"odh-model-controller-7998c67dc4-xbsqz\" (UID: \"c5f4c650-01f9-4513-9951-cf1637e933be\") " pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:05.118674 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.118681 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g9fl\" (UniqueName: \"kubernetes.io/projected/c5f4c650-01f9-4513-9951-cf1637e933be-kube-api-access-6g9fl\") pod \"odh-model-controller-7998c67dc4-xbsqz\" (UID: \"c5f4c650-01f9-4513-9951-cf1637e933be\") " pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:05.118986 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:05.118770 2556 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 10 19:14:05.118986 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:05.118884 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f4c650-01f9-4513-9951-cf1637e933be-cert podName:c5f4c650-01f9-4513-9951-cf1637e933be nodeName:}" failed. No retries permitted until 2026-07-10 19:14:05.618845554 +0000 UTC m=+397.635303918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c5f4c650-01f9-4513-9951-cf1637e933be-cert") pod "odh-model-controller-7998c67dc4-xbsqz" (UID: "c5f4c650-01f9-4513-9951-cf1637e933be") : secret "odh-model-controller-webhook-cert" not found Jul 10 19:14:05.132216 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.132184 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g9fl\" (UniqueName: \"kubernetes.io/projected/c5f4c650-01f9-4513-9951-cf1637e933be-kube-api-access-6g9fl\") pod \"odh-model-controller-7998c67dc4-xbsqz\" (UID: \"c5f4c650-01f9-4513-9951-cf1637e933be\") " pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:05.219512 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.219404 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/901caad2-0055-499e-a5d5-77e23ed88deb-tls-certs\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:05.222121 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.222077 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/901caad2-0055-499e-a5d5-77e23ed88deb-tls-certs\") pod \"kube-auth-proxy-967644b46-jmc4g\" (UID: \"901caad2-0055-499e-a5d5-77e23ed88deb\") " pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:05.468701 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.468651 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" Jul 10 19:14:05.624086 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.623989 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5f4c650-01f9-4513-9951-cf1637e933be-cert\") pod \"odh-model-controller-7998c67dc4-xbsqz\" (UID: \"c5f4c650-01f9-4513-9951-cf1637e933be\") " pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:05.627350 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.627316 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5f4c650-01f9-4513-9951-cf1637e933be-cert\") pod \"odh-model-controller-7998c67dc4-xbsqz\" (UID: \"c5f4c650-01f9-4513-9951-cf1637e933be\") " pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:05.641026 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.640999 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-967644b46-jmc4g"] Jul 10 19:14:05.642496 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:14:05.642467 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901caad2_0055_499e_a5d5_77e23ed88deb.slice/crio-363e1e98619e5c7dd74b647e3f2102f25031dce5f1f56dfa1e0775a925e8c461 WatchSource:0}: Error finding container 363e1e98619e5c7dd74b647e3f2102f25031dce5f1f56dfa1e0775a925e8c461: Status 404 returned error can't find the container with id 363e1e98619e5c7dd74b647e3f2102f25031dce5f1f56dfa1e0775a925e8c461 Jul 10 19:14:05.884299 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.884261 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:05.967389 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.967164 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hf7fn"] Jul 10 19:14:05.971307 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.971275 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:05.974824 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.974792 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 10 19:14:05.975469 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.975445 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-9qnn4\"" Jul 10 19:14:05.988696 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:05.988666 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hf7fn"] Jul 10 19:14:06.028265 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.028230 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be88e0b7-b17f-46bb-a160-fe4f7f3294f9-tls-certs\") pod \"model-serving-api-54d89bd5fc-hf7fn\" (UID: \"be88e0b7-b17f-46bb-a160-fe4f7f3294f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:06.028565 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.028541 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlfp7\" (UniqueName: \"kubernetes.io/projected/be88e0b7-b17f-46bb-a160-fe4f7f3294f9-kube-api-access-nlfp7\") pod \"model-serving-api-54d89bd5fc-hf7fn\" (UID: \"be88e0b7-b17f-46bb-a160-fe4f7f3294f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:06.058642 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.058595 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xbsqz"] Jul 10 19:14:06.061735 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:14:06.061494 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5f4c650_01f9_4513_9951_cf1637e933be.slice/crio-5ebe4cc1758265f2cb87ec760d71726e29595f02b26e9761a2bbeb4353422e25 WatchSource:0}: Error finding container 5ebe4cc1758265f2cb87ec760d71726e29595f02b26e9761a2bbeb4353422e25: Status 404 returned error can't find the container with id 5ebe4cc1758265f2cb87ec760d71726e29595f02b26e9761a2bbeb4353422e25 Jul 10 19:14:06.129736 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.129701 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be88e0b7-b17f-46bb-a160-fe4f7f3294f9-tls-certs\") pod \"model-serving-api-54d89bd5fc-hf7fn\" (UID: \"be88e0b7-b17f-46bb-a160-fe4f7f3294f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:06.129949 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.129743 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlfp7\" (UniqueName: \"kubernetes.io/projected/be88e0b7-b17f-46bb-a160-fe4f7f3294f9-kube-api-access-nlfp7\") pod \"model-serving-api-54d89bd5fc-hf7fn\" (UID: \"be88e0b7-b17f-46bb-a160-fe4f7f3294f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:06.129949 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:06.129905 2556 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 10 19:14:06.130065 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:06.129982 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be88e0b7-b17f-46bb-a160-fe4f7f3294f9-tls-certs podName:be88e0b7-b17f-46bb-a160-fe4f7f3294f9 nodeName:}" failed. No retries permitted until 2026-07-10 19:14:06.629960181 +0000 UTC m=+398.646418529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/be88e0b7-b17f-46bb-a160-fe4f7f3294f9-tls-certs") pod "model-serving-api-54d89bd5fc-hf7fn" (UID: "be88e0b7-b17f-46bb-a160-fe4f7f3294f9") : secret "model-serving-api-tls" not found Jul 10 19:14:06.154005 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.153321 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlfp7\" (UniqueName: \"kubernetes.io/projected/be88e0b7-b17f-46bb-a160-fe4f7f3294f9-kube-api-access-nlfp7\") pod \"model-serving-api-54d89bd5fc-hf7fn\" (UID: \"be88e0b7-b17f-46bb-a160-fe4f7f3294f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:06.236126 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.236090 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g"] Jul 10 19:14:06.239900 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.239854 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.247428 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.247230 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 19:14:06.247428 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.247289 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 19:14:06.247917 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.247717 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fmmdt\"" Jul 10 19:14:06.247917 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.247733 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 19:14:06.247917 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.247773 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 19:14:06.268245 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.268204 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g"] Jul 10 19:14:06.322641 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.322598 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" event={"ID":"901caad2-0055-499e-a5d5-77e23ed88deb","Type":"ContainerStarted","Data":"363e1e98619e5c7dd74b647e3f2102f25031dce5f1f56dfa1e0775a925e8c461"} Jul 10 19:14:06.324072 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.324040 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" event={"ID":"c5f4c650-01f9-4513-9951-cf1637e933be","Type":"ContainerStarted","Data":"5ebe4cc1758265f2cb87ec760d71726e29595f02b26e9761a2bbeb4353422e25"} Jul 10 19:14:06.331681 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.331645 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.331681 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.331685 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t97l\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-kube-api-access-9t97l\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.331962 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.331706 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c9abd1-2819-4af1-a376-bdb23379fa1f-local-certs\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.331962 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.331721 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-cacerts\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.332091 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.331993 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.332091 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.332037 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.332179 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.332105 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-token\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.433028 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.432932 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.433028 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.432988 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.433232 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.433047 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-token\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.433232 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.433113 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.433232 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.433138 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t97l\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-kube-api-access-9t97l\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.433569 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.433537 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c9abd1-2819-4af1-a376-bdb23379fa1f-local-certs\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.433710 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.433585 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-cacerts\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.434132 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.434093 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.436983 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.436952 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c9abd1-2819-4af1-a376-bdb23379fa1f-local-certs\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.437115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.436954 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-cacerts\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.437115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.437009 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.437115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.437009 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.443440 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.443410 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-token\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.443772 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.443748 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t97l\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-kube-api-access-9t97l\") pod \"istiod-openshift-gateway-7574459656-9ch7g\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.552123 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.552082 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:06.635645 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.635603 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be88e0b7-b17f-46bb-a160-fe4f7f3294f9-tls-certs\") pod \"model-serving-api-54d89bd5fc-hf7fn\" (UID: \"be88e0b7-b17f-46bb-a160-fe4f7f3294f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:06.639215 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.639185 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be88e0b7-b17f-46bb-a160-fe4f7f3294f9-tls-certs\") pod \"model-serving-api-54d89bd5fc-hf7fn\" (UID: \"be88e0b7-b17f-46bb-a160-fe4f7f3294f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:06.755134 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.755053 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g"] Jul 10 19:14:06.758346 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:14:06.758309 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c9abd1_2819_4af1_a376_bdb23379fa1f.slice/crio-6bda55053415c20ba37b6f88597dfc33d8fc30024daaa490269dc8a11113c1f5 WatchSource:0}: Error finding container 6bda55053415c20ba37b6f88597dfc33d8fc30024daaa490269dc8a11113c1f5: Status 404 returned error can't find the container with id 6bda55053415c20ba37b6f88597dfc33d8fc30024daaa490269dc8a11113c1f5 Jul 10 19:14:06.889912 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:06.889663 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:07.097563 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:07.097527 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hf7fn"] Jul 10 19:14:07.333343 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:07.333216 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" event={"ID":"44c9abd1-2819-4af1-a376-bdb23379fa1f","Type":"ContainerStarted","Data":"6bda55053415c20ba37b6f88597dfc33d8fc30024daaa490269dc8a11113c1f5"} Jul 10 19:14:07.337309 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:07.337262 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" event={"ID":"be88e0b7-b17f-46bb-a160-fe4f7f3294f9","Type":"ContainerStarted","Data":"e0cdb0faae6bf8832f4a6f6fec2eca00c2d0929bf47e42c5c3d48b1dae2f35bb"} Jul 10 19:14:11.755163 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:11.755096 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:14:11.755491 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:11.755208 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:14:12.388458 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.388417 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" event={"ID":"be88e0b7-b17f-46bb-a160-fe4f7f3294f9","Type":"ContainerStarted","Data":"5244f5d49caf023c3e7207637a671bbe3ae7af03ab37d942739330fa7e538f64"} Jul 10 19:14:12.388668 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.388559 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:12.390850 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.390817 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" event={"ID":"901caad2-0055-499e-a5d5-77e23ed88deb","Type":"ContainerStarted","Data":"7f58d10b705fe28f5ed5610bd090fa00b5f83e41242399d0019ea91ee2527109"} Jul 10 19:14:12.392781 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.392749 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" event={"ID":"44c9abd1-2819-4af1-a376-bdb23379fa1f","Type":"ContainerStarted","Data":"ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7"} Jul 10 19:14:12.393017 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.392995 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:12.394530 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.394506 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" event={"ID":"c5f4c650-01f9-4513-9951-cf1637e933be","Type":"ContainerStarted","Data":"a4b5b42463ad462b9f2418fbccf0260ee39837cafba889c55e55d51188ae281a"} Jul 10 19:14:12.394672 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.394636 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:12.395371 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.395348 2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-9ch7g container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 19:14:12.395485 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.395403 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" podUID="44c9abd1-2819-4af1-a376-bdb23379fa1f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 19:14:12.413593 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.413518 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" podStartSLOduration=2.763915319 podStartE2EDuration="7.413496333s" totalImagesPullingTime="4.649581014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:14:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:14:12.409976252 +0000 UTC m=+404.426434615" watchObservedRunningTime="2026-07-10 19:14:12.413496333 +0000 UTC m=+404.429954695" Jul 10 19:14:12.434152 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.434080 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-967644b46-jmc4g" podStartSLOduration=2.323621927 podStartE2EDuration="8.434058388s" totalImagesPullingTime="6.110436461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:14:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:14:12.430257933 +0000 UTC m=+404.446716305" watchObservedRunningTime="2026-07-10 19:14:12.434058388 +0000 UTC m=+404.450516760" Jul 10 19:14:12.456776 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.456713 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" podStartSLOduration=1.464164466 podStartE2EDuration="6.456667479s" totalImagesPullingTime="4.992503013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:14:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:14:12.45537639 +0000 UTC m=+404.471834789" watchObservedRunningTime="2026-07-10 19:14:12.456667479 +0000 UTC m=+404.473125849" Jul 10 19:14:12.488261 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:12.488193 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" podStartSLOduration=2.799982386 podStartE2EDuration="8.488166237s" totalImagesPullingTime="5.688183851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:14:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:14:12.48575296 +0000 UTC m=+404.502211324" watchObservedRunningTime="2026-07-10 19:14:12.488166237 +0000 UTC m=+404.504624617" Jul 10 19:14:13.400780 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:13.400747 2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-9ch7g container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 19:14:13.401274 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:13.400799 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" podUID="44c9abd1-2819-4af1-a376-bdb23379fa1f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 19:14:14.017489 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.017447 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f"] Jul 10 19:14:14.021188 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.021161 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.025235 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.025211 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5sw4m\"" Jul 10 19:14:14.040041 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.040004 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f"] Jul 10 19:14:14.114243 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.114190 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.114426 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.114249 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.114426 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.114284 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.114426 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.114345 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.114426 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.114395 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.114426 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.114421 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj2pq\" (UniqueName: \"kubernetes.io/projected/00e32cf7-d620-4529-b1f9-a77357c329ef-kube-api-access-pj2pq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.114723 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.114509 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.114723 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.114560 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00e32cf7-d620-4529-b1f9-a77357c329ef-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.114723 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.114607 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215215 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215168 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215228 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj2pq\" (UniqueName: \"kubernetes.io/projected/00e32cf7-d620-4529-b1f9-a77357c329ef-kube-api-access-pj2pq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215274 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215308 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00e32cf7-d620-4529-b1f9-a77357c329ef-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215346 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215406 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215971 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215439 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215971 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215466 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215971 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215507 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215971 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215732 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.215971 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.215813 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.216210 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.216029 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.216210 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.216128 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00e32cf7-d620-4529-b1f9-a77357c329ef-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.216210 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.216156 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.218085 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.218057 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.218209 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.218190 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.236327 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.236296 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00e32cf7-d620-4529-b1f9-a77357c329ef-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.236505 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.236344 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj2pq\" (UniqueName: \"kubernetes.io/projected/00e32cf7-d620-4529-b1f9-a77357c329ef-kube-api-access-pj2pq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gc88f\" (UID: \"00e32cf7-d620-4529-b1f9-a77357c329ef\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.332754 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.332663 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:14.485762 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:14.485726 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f"] Jul 10 19:14:14.487760 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:14:14.487735 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e32cf7_d620_4529_b1f9_a77357c329ef.slice/crio-ceb4bfa8e4811f2a2b5730556901f1bdacbed2ba03116c5ed73ce4b739d659a4 WatchSource:0}: Error finding container ceb4bfa8e4811f2a2b5730556901f1bdacbed2ba03116c5ed73ce4b739d659a4: Status 404 returned error can't find the container with id ceb4bfa8e4811f2a2b5730556901f1bdacbed2ba03116c5ed73ce4b739d659a4 Jul 10 19:14:15.415276 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:15.415240 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" event={"ID":"00e32cf7-d620-4529-b1f9-a77357c329ef","Type":"ContainerStarted","Data":"ceb4bfa8e4811f2a2b5730556901f1bdacbed2ba03116c5ed73ce4b739d659a4"} Jul 10 19:14:16.401558 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:16.401502 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:17.098679 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.098619 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:14:17.098817 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.098718 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:14:17.098817 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.098761 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:14:17.254713 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.254676 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh"] Jul 10 19:14:17.258294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.258275 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:17.261163 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.261132 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 19:14:17.261338 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.261145 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nghcs\"" Jul 10 19:14:17.264650 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.264626 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr"] Jul 10 19:14:17.268492 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.268459 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:17.271148 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.271112 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 19:14:17.271288 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.271264 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-89jr4\"" Jul 10 19:14:17.277644 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.277504 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh"] Jul 10 19:14:17.286587 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.286542 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr"] Jul 10 19:14:17.349829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.349724 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b63c270-508e-4943-82f3-61a1feae75fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-8r9zh\" (UID: \"3b63c270-508e-4943-82f3-61a1feae75fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:17.349829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.349775 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnbzt\" (UniqueName: \"kubernetes.io/projected/3b63c270-508e-4943-82f3-61a1feae75fd-kube-api-access-tnbzt\") pod \"kserve-controller-manager-d6f4b65b8-8r9zh\" (UID: \"3b63c270-508e-4943-82f3-61a1feae75fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:17.350090 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.349962 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhjhm\" (UniqueName: \"kubernetes.io/projected/69647bae-3f04-4178-a377-8a8f34dd2fdc-kube-api-access-hhjhm\") pod \"llmisvc-controller-manager-777c6bd7d5-qs4zr\" (UID: \"69647bae-3f04-4178-a377-8a8f34dd2fdc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:17.350090 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.350005 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69647bae-3f04-4178-a377-8a8f34dd2fdc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qs4zr\" (UID: \"69647bae-3f04-4178-a377-8a8f34dd2fdc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:17.430818 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.430777 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" event={"ID":"00e32cf7-d620-4529-b1f9-a77357c329ef","Type":"ContainerStarted","Data":"89357933779b19d3fbb592b4d641423e7433f00ad00e8fafc22dd2ba206ac817"} Jul 10 19:14:17.450687 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.450649 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b63c270-508e-4943-82f3-61a1feae75fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-8r9zh\" (UID: \"3b63c270-508e-4943-82f3-61a1feae75fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:17.450687 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.450690 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnbzt\" (UniqueName: \"kubernetes.io/projected/3b63c270-508e-4943-82f3-61a1feae75fd-kube-api-access-tnbzt\") pod \"kserve-controller-manager-d6f4b65b8-8r9zh\" (UID: \"3b63c270-508e-4943-82f3-61a1feae75fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:17.450997 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.450761 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhjhm\" (UniqueName: \"kubernetes.io/projected/69647bae-3f04-4178-a377-8a8f34dd2fdc-kube-api-access-hhjhm\") pod \"llmisvc-controller-manager-777c6bd7d5-qs4zr\" (UID: \"69647bae-3f04-4178-a377-8a8f34dd2fdc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:17.450997 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.450784 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69647bae-3f04-4178-a377-8a8f34dd2fdc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qs4zr\" (UID: \"69647bae-3f04-4178-a377-8a8f34dd2fdc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:17.450997 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:17.450843 2556 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 10 19:14:17.450997 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:17.450910 2556 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 10 19:14:17.450997 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:17.450965 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69647bae-3f04-4178-a377-8a8f34dd2fdc-cert podName:69647bae-3f04-4178-a377-8a8f34dd2fdc nodeName:}" failed. No retries permitted until 2026-07-10 19:14:17.950944721 +0000 UTC m=+409.967403072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/69647bae-3f04-4178-a377-8a8f34dd2fdc-cert") pod "llmisvc-controller-manager-777c6bd7d5-qs4zr" (UID: "69647bae-3f04-4178-a377-8a8f34dd2fdc") : secret "llmisvc-webhook-server-cert" not found Jul 10 19:14:17.450997 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:17.450983 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b63c270-508e-4943-82f3-61a1feae75fd-cert podName:3b63c270-508e-4943-82f3-61a1feae75fd nodeName:}" failed. No retries permitted until 2026-07-10 19:14:17.95097373 +0000 UTC m=+409.967432080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3b63c270-508e-4943-82f3-61a1feae75fd-cert") pod "kserve-controller-manager-d6f4b65b8-8r9zh" (UID: "3b63c270-508e-4943-82f3-61a1feae75fd") : secret "kserve-webhook-server-cert" not found Jul 10 19:14:17.463948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.463920 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnbzt\" (UniqueName: \"kubernetes.io/projected/3b63c270-508e-4943-82f3-61a1feae75fd-kube-api-access-tnbzt\") pod \"kserve-controller-manager-d6f4b65b8-8r9zh\" (UID: \"3b63c270-508e-4943-82f3-61a1feae75fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:17.473773 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.473741 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhjhm\" (UniqueName: \"kubernetes.io/projected/69647bae-3f04-4178-a377-8a8f34dd2fdc-kube-api-access-hhjhm\") pod \"llmisvc-controller-manager-777c6bd7d5-qs4zr\" (UID: \"69647bae-3f04-4178-a377-8a8f34dd2fdc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:17.504684 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.504630 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" podStartSLOduration=1.8962724469999999 podStartE2EDuration="4.504613083s" totalImagesPullingTime="2.608340636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:14:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:14:17.503128385 +0000 UTC m=+409.519586756" watchObservedRunningTime="2026-07-10 19:14:17.504613083 +0000 UTC m=+409.521071453" Jul 10 19:14:17.955966 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.955922 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b63c270-508e-4943-82f3-61a1feae75fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-8r9zh\" (UID: \"3b63c270-508e-4943-82f3-61a1feae75fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:17.956142 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.956058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69647bae-3f04-4178-a377-8a8f34dd2fdc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qs4zr\" (UID: \"69647bae-3f04-4178-a377-8a8f34dd2fdc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:17.958638 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.958611 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69647bae-3f04-4178-a377-8a8f34dd2fdc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qs4zr\" (UID: \"69647bae-3f04-4178-a377-8a8f34dd2fdc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:17.958638 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:17.958623 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b63c270-508e-4943-82f3-61a1feae75fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-8r9zh\" (UID: \"3b63c270-508e-4943-82f3-61a1feae75fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:18.170660 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.170612 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:18.180829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.180800 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:18.333548 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.333451 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:18.352247 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.351764 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:18.394184 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.394154 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh"] Jul 10 19:14:18.395702 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:14:18.395675 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b63c270_508e_4943_82f3_61a1feae75fd.slice/crio-584a53c16d590e1d14bdf50890b08fa230df05d348e01aa9aa6035e2d642a8e1 WatchSource:0}: Error finding container 584a53c16d590e1d14bdf50890b08fa230df05d348e01aa9aa6035e2d642a8e1: Status 404 returned error can't find the container with id 584a53c16d590e1d14bdf50890b08fa230df05d348e01aa9aa6035e2d642a8e1 Jul 10 19:14:18.419210 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.419184 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr"] Jul 10 19:14:18.423779 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:14:18.423747 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod69647bae_3f04_4178_a377_8a8f34dd2fdc.slice/crio-6d7e9bc29537e09a337fdb1f7153cad90c4bed1389d7e5e43590b9080c08e680 WatchSource:0}: Error finding container 6d7e9bc29537e09a337fdb1f7153cad90c4bed1389d7e5e43590b9080c08e680: Status 404 returned error can't find the container with id 6d7e9bc29537e09a337fdb1f7153cad90c4bed1389d7e5e43590b9080c08e680 Jul 10 19:14:18.437583 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.437545 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" event={"ID":"3b63c270-508e-4943-82f3-61a1feae75fd","Type":"ContainerStarted","Data":"584a53c16d590e1d14bdf50890b08fa230df05d348e01aa9aa6035e2d642a8e1"} Jul 10 19:14:18.438972 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.438944 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" event={"ID":"69647bae-3f04-4178-a377-8a8f34dd2fdc","Type":"ContainerStarted","Data":"6d7e9bc29537e09a337fdb1f7153cad90c4bed1389d7e5e43590b9080c08e680"} Jul 10 19:14:18.439167 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.439152 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:18.440179 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:18.440160 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gc88f" Jul 10 19:14:22.465158 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:22.465067 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" event={"ID":"3b63c270-508e-4943-82f3-61a1feae75fd","Type":"ContainerStarted","Data":"3371c025316d11516fc6a9190f427287492bad441318757d098acc0457401d35"} Jul 10 19:14:22.465608 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:22.465182 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:14:22.466763 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:22.466730 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" event={"ID":"69647bae-3f04-4178-a377-8a8f34dd2fdc","Type":"ContainerStarted","Data":"6a702f25fe5c6a3c12f1343d40abd1e8c1ff1fb550728035c918e22e0f5b7abc"} Jul 10 19:14:22.466904 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:22.466848 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:22.483427 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:22.483371 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" podStartSLOduration=1.732154731 podStartE2EDuration="5.483353224s" totalImagesPullingTime="3.751198493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:14:22.482356095 +0000 UTC m=+414.498814469" watchObservedRunningTime="2026-07-10 19:14:22.483353224 +0000 UTC m=+414.499811594" Jul 10 19:14:22.500629 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:22.500570 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" podStartSLOduration=1.771585942 podStartE2EDuration="5.500550382s" totalImagesPullingTime="3.72896444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:14:22.500328714 +0000 UTC m=+414.516787086" watchObservedRunningTime="2026-07-10 19:14:22.500550382 +0000 UTC m=+414.517008753" Jul 10 19:14:23.402798 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:23.402764 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xbsqz" Jul 10 19:14:23.404425 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:23.404402 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hf7fn" Jul 10 19:14:27.485527 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.485481 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g"] Jul 10 19:14:27.486030 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.485734 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" podUID="44c9abd1-2819-4af1-a376-bdb23379fa1f" containerName="discovery" containerID="cri-o://ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7" gracePeriod=30 Jul 10 19:14:27.763521 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.763493 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:27.849715 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.849676 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-token\") pod \"44c9abd1-2819-4af1-a376-bdb23379fa1f\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " Jul 10 19:14:27.849912 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.849777 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-dns-cert\") pod \"44c9abd1-2819-4af1-a376-bdb23379fa1f\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " Jul 10 19:14:27.849912 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.849814 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-cacerts\") pod \"44c9abd1-2819-4af1-a376-bdb23379fa1f\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " Jul 10 19:14:27.849912 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.849842 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c9abd1-2819-4af1-a376-bdb23379fa1f-local-certs\") pod \"44c9abd1-2819-4af1-a376-bdb23379fa1f\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " Jul 10 19:14:27.849912 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.849905 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-kubeconfig\") pod \"44c9abd1-2819-4af1-a376-bdb23379fa1f\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " Jul 10 19:14:27.850151 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.849969 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t97l\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-kube-api-access-9t97l\") pod \"44c9abd1-2819-4af1-a376-bdb23379fa1f\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " Jul 10 19:14:27.850151 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.850004 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-ca-configmap\") pod \"44c9abd1-2819-4af1-a376-bdb23379fa1f\" (UID: \"44c9abd1-2819-4af1-a376-bdb23379fa1f\") " Jul 10 19:14:27.850439 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.850406 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "44c9abd1-2819-4af1-a376-bdb23379fa1f" (UID: "44c9abd1-2819-4af1-a376-bdb23379fa1f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:14:27.850719 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.850691 2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-ca-configmap\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:14:27.852641 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.852600 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "44c9abd1-2819-4af1-a376-bdb23379fa1f" (UID: "44c9abd1-2819-4af1-a376-bdb23379fa1f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:14:27.852641 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.852620 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-cacerts" (OuterVolumeSpecName: "cacerts") pod "44c9abd1-2819-4af1-a376-bdb23379fa1f" (UID: "44c9abd1-2819-4af1-a376-bdb23379fa1f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:14:27.853293 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.853262 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "44c9abd1-2819-4af1-a376-bdb23379fa1f" (UID: "44c9abd1-2819-4af1-a376-bdb23379fa1f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:14:27.853399 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.853330 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-kube-api-access-9t97l" (OuterVolumeSpecName: "kube-api-access-9t97l") pod "44c9abd1-2819-4af1-a376-bdb23379fa1f" (UID: "44c9abd1-2819-4af1-a376-bdb23379fa1f"). InnerVolumeSpecName "kube-api-access-9t97l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:14:27.853399 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.853330 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-token" (OuterVolumeSpecName: "istio-token") pod "44c9abd1-2819-4af1-a376-bdb23379fa1f" (UID: "44c9abd1-2819-4af1-a376-bdb23379fa1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:14:27.854359 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.854340 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44c9abd1-2819-4af1-a376-bdb23379fa1f-local-certs" (OuterVolumeSpecName: "local-certs") pod "44c9abd1-2819-4af1-a376-bdb23379fa1f" (UID: "44c9abd1-2819-4af1-a376-bdb23379fa1f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:14:27.952074 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.952029 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-token\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:14:27.952074 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.952073 2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-csr-dns-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:14:27.952280 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.952089 2556 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-cacerts\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:14:27.952280 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.952103 2556 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c9abd1-2819-4af1-a376-bdb23379fa1f-local-certs\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:14:27.952280 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.952118 2556 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44c9abd1-2819-4af1-a376-bdb23379fa1f-istio-kubeconfig\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:14:27.952280 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:27.952131 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t97l\" (UniqueName: \"kubernetes.io/projected/44c9abd1-2819-4af1-a376-bdb23379fa1f-kube-api-access-9t97l\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:14:28.499692 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:28.499656 2556 generic.go:356] "Generic (PLEG): container finished" podID="44c9abd1-2819-4af1-a376-bdb23379fa1f" containerID="ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7" exitCode=0 Jul 10 19:14:28.500209 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:28.499712 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" Jul 10 19:14:28.500209 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:28.499743 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" event={"ID":"44c9abd1-2819-4af1-a376-bdb23379fa1f","Type":"ContainerDied","Data":"ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7"} Jul 10 19:14:28.500209 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:28.499809 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g" event={"ID":"44c9abd1-2819-4af1-a376-bdb23379fa1f","Type":"ContainerDied","Data":"6bda55053415c20ba37b6f88597dfc33d8fc30024daaa490269dc8a11113c1f5"} Jul 10 19:14:28.500209 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:28.499852 2556 scope.go:117] "RemoveContainer" containerID="ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7" Jul 10 19:14:28.509263 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:28.509246 2556 scope.go:117] "RemoveContainer" containerID="ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7" Jul 10 19:14:28.509535 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:14:28.509519 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7\": container with ID starting with ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7 not found: ID does not exist" containerID="ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7" Jul 10 19:14:28.509579 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:28.509546 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7"} err="failed to get container status \"ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7\": rpc error: code = NotFound desc = could not find container \"ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7\": container with ID starting with ad470315ada635c2036663c3fcf8c306fdc5af396f5dc9dd9309eeb6f0be96f7 not found: ID does not exist" Jul 10 19:14:28.535663 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:28.535621 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g"] Jul 10 19:14:28.539215 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:28.539162 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9ch7g"] Jul 10 19:14:30.518648 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:30.518606 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44c9abd1-2819-4af1-a376-bdb23379fa1f" path="/var/lib/kubelet/pods/44c9abd1-2819-4af1-a376-bdb23379fa1f/volumes" Jul 10 19:14:53.473996 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:53.473908 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qs4zr" Jul 10 19:14:53.477314 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:14:53.477288 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8r9zh" Jul 10 19:15:07.124296 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.124263 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg"] Jul 10 19:15:07.124709 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.124640 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44c9abd1-2819-4af1-a376-bdb23379fa1f" containerName="discovery" Jul 10 19:15:07.124709 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.124653 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="44c9abd1-2819-4af1-a376-bdb23379fa1f" containerName="discovery" Jul 10 19:15:07.124842 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.124716 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="44c9abd1-2819-4af1-a376-bdb23379fa1f" containerName="discovery" Jul 10 19:15:07.131465 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.131442 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.135482 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.135451 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 19:15:07.135633 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.135508 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gqrrr\"" Jul 10 19:15:07.135846 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.135803 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 19:15:07.139183 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.139157 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg"] Jul 10 19:15:07.207335 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.207298 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4krq\" (UniqueName: \"kubernetes.io/projected/3bbcf0f5-4989-4d02-9734-db81ee444a40-kube-api-access-r4krq\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.207335 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.207345 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.207586 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.207371 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.225399 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.225367 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b"] Jul 10 19:15:07.228193 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.228166 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.238337 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.238308 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b"] Jul 10 19:15:07.307942 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.307907 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4krq\" (UniqueName: \"kubernetes.io/projected/3bbcf0f5-4989-4d02-9734-db81ee444a40-kube-api-access-r4krq\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.308129 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.307955 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.308129 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.307995 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.308129 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.308037 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.308129 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.308117 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tg6k\" (UniqueName: \"kubernetes.io/projected/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-kube-api-access-4tg6k\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.308301 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.308144 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.308427 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.308405 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.308462 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.308445 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.316517 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.316472 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq"] Jul 10 19:15:07.319464 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.319440 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.323005 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.322980 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4krq\" (UniqueName: \"kubernetes.io/projected/3bbcf0f5-4989-4d02-9734-db81ee444a40-kube-api-access-r4krq\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.328628 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.328598 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq"] Jul 10 19:15:07.408710 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.408615 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.408710 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.408688 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tg6k\" (UniqueName: \"kubernetes.io/projected/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-kube-api-access-4tg6k\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.408710 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.408710 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.409034 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.408727 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts7s6\" (UniqueName: \"kubernetes.io/projected/223d0dff-a6c3-48cc-b564-afac8ead5350-kube-api-access-ts7s6\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.409034 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.408747 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.409034 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.408827 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.409191 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.409148 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.409191 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.409184 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.415615 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.415584 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9"] Jul 10 19:15:07.418636 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.418614 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.423592 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.423560 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tg6k\" (UniqueName: \"kubernetes.io/projected/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-kube-api-access-4tg6k\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.427563 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.427535 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9"] Jul 10 19:15:07.443244 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.443211 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:07.510495 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.510449 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.510680 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.510567 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.510680 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.510601 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tchg6\" (UniqueName: \"kubernetes.io/projected/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-kube-api-access-tchg6\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.510680 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.510656 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.510852 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.510686 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts7s6\" (UniqueName: \"kubernetes.io/projected/223d0dff-a6c3-48cc-b564-afac8ead5350-kube-api-access-ts7s6\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.510852 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.510722 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.511148 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.511101 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.511282 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.511213 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.521439 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.521398 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts7s6\" (UniqueName: \"kubernetes.io/projected/223d0dff-a6c3-48cc-b564-afac8ead5350-kube-api-access-ts7s6\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.538247 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.538207 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:07.594770 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.594731 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg"] Jul 10 19:15:07.596147 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:07.596101 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bbcf0f5_4989_4d02_9734_db81ee444a40.slice/crio-d4e1b64899deb9946a8ecad427434c35344b2d92df90b9693be1742566293096 WatchSource:0}: Error finding container d4e1b64899deb9946a8ecad427434c35344b2d92df90b9693be1742566293096: Status 404 returned error can't find the container with id d4e1b64899deb9946a8ecad427434c35344b2d92df90b9693be1742566293096 Jul 10 19:15:07.611848 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.611806 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tchg6\" (UniqueName: \"kubernetes.io/projected/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-kube-api-access-tchg6\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.612019 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.611906 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.612019 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.611966 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.612337 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.612312 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.612337 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.612325 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.621435 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.621352 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tchg6\" (UniqueName: \"kubernetes.io/projected/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-kube-api-access-tchg6\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.639331 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.638907 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:07.699746 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.699589 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" event={"ID":"3bbcf0f5-4989-4d02-9734-db81ee444a40","Type":"ContainerStarted","Data":"d4e1b64899deb9946a8ecad427434c35344b2d92df90b9693be1742566293096"} Jul 10 19:15:07.715826 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.715774 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b"] Jul 10 19:15:07.719236 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:07.719206 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4be61a_c40f_42eb_a7bb_cbfbb209424d.slice/crio-2519ac4f1d73ef5042616b7e51e5a1be002a80935b7a24f60e9969d510f5782f WatchSource:0}: Error finding container 2519ac4f1d73ef5042616b7e51e5a1be002a80935b7a24f60e9969d510f5782f: Status 404 returned error can't find the container with id 2519ac4f1d73ef5042616b7e51e5a1be002a80935b7a24f60e9969d510f5782f Jul 10 19:15:07.738036 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.738004 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:07.811678 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.811638 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq"] Jul 10 19:15:07.813297 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:07.813261 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223d0dff_a6c3_48cc_b564_afac8ead5350.slice/crio-dd09b4147a3f79b0d784ea43bbc588c7528e5f05c56833e678a536726e9924d9 WatchSource:0}: Error finding container dd09b4147a3f79b0d784ea43bbc588c7528e5f05c56833e678a536726e9924d9: Status 404 returned error can't find the container with id dd09b4147a3f79b0d784ea43bbc588c7528e5f05c56833e678a536726e9924d9 Jul 10 19:15:07.908819 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:07.908793 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9"] Jul 10 19:15:07.941429 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:07.941398 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9140f2e3_f0dd_4cf8_aa8e_2aa89a403b5f.slice/crio-8f65eb69c76111786cede9cbe6ee5ca06ea12a00c22d77b943647f2fb3a4d7d5 WatchSource:0}: Error finding container 8f65eb69c76111786cede9cbe6ee5ca06ea12a00c22d77b943647f2fb3a4d7d5: Status 404 returned error can't find the container with id 8f65eb69c76111786cede9cbe6ee5ca06ea12a00c22d77b943647f2fb3a4d7d5 Jul 10 19:15:08.706497 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.706464 2556 generic.go:356] "Generic (PLEG): container finished" podID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerID="69986aaa548dd002bbc13a0417657a10f14b05648c7d47656e1838c3a14f814b" exitCode=0 Jul 10 19:15:08.707000 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.706550 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" event={"ID":"3bbcf0f5-4989-4d02-9734-db81ee444a40","Type":"ContainerDied","Data":"69986aaa548dd002bbc13a0417657a10f14b05648c7d47656e1838c3a14f814b"} Jul 10 19:15:08.708276 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.708255 2556 generic.go:356] "Generic (PLEG): container finished" podID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerID="b84e9c0ad06814523e0dfcf809c6496c8c25a377959c8758f409a4bffad429de" exitCode=0 Jul 10 19:15:08.708388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.708372 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" event={"ID":"4a4be61a-c40f-42eb-a7bb-cbfbb209424d","Type":"ContainerDied","Data":"b84e9c0ad06814523e0dfcf809c6496c8c25a377959c8758f409a4bffad429de"} Jul 10 19:15:08.708449 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.708418 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" event={"ID":"4a4be61a-c40f-42eb-a7bb-cbfbb209424d","Type":"ContainerStarted","Data":"2519ac4f1d73ef5042616b7e51e5a1be002a80935b7a24f60e9969d510f5782f"} Jul 10 19:15:08.709979 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.709947 2556 generic.go:356] "Generic (PLEG): container finished" podID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerID="1db1e34d25e1895685e739b0057707534efcb3e72fbabc9ef4936a3ff4222f84" exitCode=0 Jul 10 19:15:08.710084 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.709977 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" event={"ID":"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f","Type":"ContainerDied","Data":"1db1e34d25e1895685e739b0057707534efcb3e72fbabc9ef4936a3ff4222f84"} Jul 10 19:15:08.710084 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.710024 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" event={"ID":"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f","Type":"ContainerStarted","Data":"8f65eb69c76111786cede9cbe6ee5ca06ea12a00c22d77b943647f2fb3a4d7d5"} Jul 10 19:15:08.711922 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.711901 2556 generic.go:356] "Generic (PLEG): container finished" podID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerID="6c0ba037e9a354996c7a3171484e504f55c8541ac89b78455b3b5f3fcdac3cd2" exitCode=0 Jul 10 19:15:08.712012 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.711950 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" event={"ID":"223d0dff-a6c3-48cc-b564-afac8ead5350","Type":"ContainerDied","Data":"6c0ba037e9a354996c7a3171484e504f55c8541ac89b78455b3b5f3fcdac3cd2"} Jul 10 19:15:08.712012 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:08.711985 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" event={"ID":"223d0dff-a6c3-48cc-b564-afac8ead5350","Type":"ContainerStarted","Data":"dd09b4147a3f79b0d784ea43bbc588c7528e5f05c56833e678a536726e9924d9"} Jul 10 19:15:09.719390 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:09.719297 2556 generic.go:356] "Generic (PLEG): container finished" podID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerID="7e48af07a15d96883a5c0d352803f9453fcdcac433386ef6bc386a27b44095e7" exitCode=0 Jul 10 19:15:09.719836 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:09.719378 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" event={"ID":"3bbcf0f5-4989-4d02-9734-db81ee444a40","Type":"ContainerDied","Data":"7e48af07a15d96883a5c0d352803f9453fcdcac433386ef6bc386a27b44095e7"} Jul 10 19:15:09.721558 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:09.721532 2556 generic.go:356] "Generic (PLEG): container finished" podID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerID="f325891d22e8efbdc3cdcc0866aa1aceb075e7de7ce1573ebf5fe4556101d5d0" exitCode=0 Jul 10 19:15:09.721644 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:09.721614 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" event={"ID":"4a4be61a-c40f-42eb-a7bb-cbfbb209424d","Type":"ContainerDied","Data":"f325891d22e8efbdc3cdcc0866aa1aceb075e7de7ce1573ebf5fe4556101d5d0"} Jul 10 19:15:10.729455 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:10.729416 2556 generic.go:356] "Generic (PLEG): container finished" podID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerID="84046d2e1d91ec7b908accdb98db05a42e7a9b08cb75a76970363c57b2fa0b8e" exitCode=0 Jul 10 19:15:10.729960 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:10.729526 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" event={"ID":"3bbcf0f5-4989-4d02-9734-db81ee444a40","Type":"ContainerDied","Data":"84046d2e1d91ec7b908accdb98db05a42e7a9b08cb75a76970363c57b2fa0b8e"} Jul 10 19:15:10.731612 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:10.731582 2556 generic.go:356] "Generic (PLEG): container finished" podID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerID="defe8ae7971e867a5689b22d700fa36b78909a1cd9ccb6cc72093959d7c6835f" exitCode=0 Jul 10 19:15:10.731733 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:10.731665 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" event={"ID":"4a4be61a-c40f-42eb-a7bb-cbfbb209424d","Type":"ContainerDied","Data":"defe8ae7971e867a5689b22d700fa36b78909a1cd9ccb6cc72093959d7c6835f"} Jul 10 19:15:10.733340 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:10.733316 2556 generic.go:356] "Generic (PLEG): container finished" podID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerID="7db8cf77c81fd1edaf3b197cf072ec7840f8021826f3aeafe1707d035266ce80" exitCode=0 Jul 10 19:15:10.733449 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:10.733400 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" event={"ID":"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f","Type":"ContainerDied","Data":"7db8cf77c81fd1edaf3b197cf072ec7840f8021826f3aeafe1707d035266ce80"} Jul 10 19:15:10.735484 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:10.735458 2556 generic.go:356] "Generic (PLEG): container finished" podID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerID="65213214bbac13dd6b8ad58529f7d3bd90cf71555a1d6273b950524457908e16" exitCode=0 Jul 10 19:15:10.735571 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:10.735513 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" event={"ID":"223d0dff-a6c3-48cc-b564-afac8ead5350","Type":"ContainerDied","Data":"65213214bbac13dd6b8ad58529f7d3bd90cf71555a1d6273b950524457908e16"} Jul 10 19:15:11.745219 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.745183 2556 generic.go:356] "Generic (PLEG): container finished" podID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerID="4a1f387f625afbc326cb5f3855ee4191d1a6e917e38f4146ec6c88da9dad5170" exitCode=0 Jul 10 19:15:11.745584 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.745252 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" event={"ID":"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f","Type":"ContainerDied","Data":"4a1f387f625afbc326cb5f3855ee4191d1a6e917e38f4146ec6c88da9dad5170"} Jul 10 19:15:11.747393 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.747369 2556 generic.go:356] "Generic (PLEG): container finished" podID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerID="879872e52fe90f51c6452b83197703711247d4024427713e2a3819e054d272ad" exitCode=0 Jul 10 19:15:11.747548 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.747454 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" event={"ID":"223d0dff-a6c3-48cc-b564-afac8ead5350","Type":"ContainerDied","Data":"879872e52fe90f51c6452b83197703711247d4024427713e2a3819e054d272ad"} Jul 10 19:15:11.893449 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.893422 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:11.934070 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.934036 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:11.951752 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.951715 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-bundle\") pod \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " Jul 10 19:15:11.951969 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.951854 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tg6k\" (UniqueName: \"kubernetes.io/projected/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-kube-api-access-4tg6k\") pod \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " Jul 10 19:15:11.951969 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.951941 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-util\") pod \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\" (UID: \"4a4be61a-c40f-42eb-a7bb-cbfbb209424d\") " Jul 10 19:15:11.952405 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.952378 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-bundle" (OuterVolumeSpecName: "bundle") pod "4a4be61a-c40f-42eb-a7bb-cbfbb209424d" (UID: "4a4be61a-c40f-42eb-a7bb-cbfbb209424d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:11.955014 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.954980 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-kube-api-access-4tg6k" (OuterVolumeSpecName: "kube-api-access-4tg6k") pod "4a4be61a-c40f-42eb-a7bb-cbfbb209424d" (UID: "4a4be61a-c40f-42eb-a7bb-cbfbb209424d"). InnerVolumeSpecName "kube-api-access-4tg6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:15:11.959186 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:11.959152 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-util" (OuterVolumeSpecName: "util") pod "4a4be61a-c40f-42eb-a7bb-cbfbb209424d" (UID: "4a4be61a-c40f-42eb-a7bb-cbfbb209424d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:12.053539 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.053433 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-util\") pod \"3bbcf0f5-4989-4d02-9734-db81ee444a40\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " Jul 10 19:15:12.053539 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.053537 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4krq\" (UniqueName: \"kubernetes.io/projected/3bbcf0f5-4989-4d02-9734-db81ee444a40-kube-api-access-r4krq\") pod \"3bbcf0f5-4989-4d02-9734-db81ee444a40\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " Jul 10 19:15:12.053762 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.053582 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-bundle\") pod \"3bbcf0f5-4989-4d02-9734-db81ee444a40\" (UID: \"3bbcf0f5-4989-4d02-9734-db81ee444a40\") " Jul 10 19:15:12.053850 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.053829 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tg6k\" (UniqueName: \"kubernetes.io/projected/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-kube-api-access-4tg6k\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:12.053951 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.053855 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-util\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:12.053951 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.053907 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a4be61a-c40f-42eb-a7bb-cbfbb209424d-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:12.054437 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.054401 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-bundle" (OuterVolumeSpecName: "bundle") pod "3bbcf0f5-4989-4d02-9734-db81ee444a40" (UID: "3bbcf0f5-4989-4d02-9734-db81ee444a40"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:12.055831 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.055803 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bbcf0f5-4989-4d02-9734-db81ee444a40-kube-api-access-r4krq" (OuterVolumeSpecName: "kube-api-access-r4krq") pod "3bbcf0f5-4989-4d02-9734-db81ee444a40" (UID: "3bbcf0f5-4989-4d02-9734-db81ee444a40"). InnerVolumeSpecName "kube-api-access-r4krq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:15:12.059439 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.059410 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-util" (OuterVolumeSpecName: "util") pod "3bbcf0f5-4989-4d02-9734-db81ee444a40" (UID: "3bbcf0f5-4989-4d02-9734-db81ee444a40"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:12.154380 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.154340 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-util\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:12.154380 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.154377 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4krq\" (UniqueName: \"kubernetes.io/projected/3bbcf0f5-4989-4d02-9734-db81ee444a40-kube-api-access-r4krq\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:12.154560 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.154392 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3bbcf0f5-4989-4d02-9734-db81ee444a40-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:12.754426 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.754400 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" Jul 10 19:15:12.754426 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.754417 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcab4zb6b" event={"ID":"4a4be61a-c40f-42eb-a7bb-cbfbb209424d","Type":"ContainerDied","Data":"2519ac4f1d73ef5042616b7e51e5a1be002a80935b7a24f60e9969d510f5782f"} Jul 10 19:15:12.754920 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.754460 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2519ac4f1d73ef5042616b7e51e5a1be002a80935b7a24f60e9969d510f5782f" Jul 10 19:15:12.756713 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.756690 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" Jul 10 19:15:12.756841 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.756753 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76fcwgg" event={"ID":"3bbcf0f5-4989-4d02-9734-db81ee444a40","Type":"ContainerDied","Data":"d4e1b64899deb9946a8ecad427434c35344b2d92df90b9693be1742566293096"} Jul 10 19:15:12.756841 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.756794 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4e1b64899deb9946a8ecad427434c35344b2d92df90b9693be1742566293096" Jul 10 19:15:12.899648 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.899618 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:12.934787 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.934757 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:12.963244 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.963214 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-util\") pod \"223d0dff-a6c3-48cc-b564-afac8ead5350\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " Jul 10 19:15:12.963427 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.963255 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts7s6\" (UniqueName: \"kubernetes.io/projected/223d0dff-a6c3-48cc-b564-afac8ead5350-kube-api-access-ts7s6\") pod \"223d0dff-a6c3-48cc-b564-afac8ead5350\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " Jul 10 19:15:12.963427 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.963332 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-bundle\") pod \"223d0dff-a6c3-48cc-b564-afac8ead5350\" (UID: \"223d0dff-a6c3-48cc-b564-afac8ead5350\") " Jul 10 19:15:12.963865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.963827 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-bundle" (OuterVolumeSpecName: "bundle") pod "223d0dff-a6c3-48cc-b564-afac8ead5350" (UID: "223d0dff-a6c3-48cc-b564-afac8ead5350"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:12.966011 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.965978 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/223d0dff-a6c3-48cc-b564-afac8ead5350-kube-api-access-ts7s6" (OuterVolumeSpecName: "kube-api-access-ts7s6") pod "223d0dff-a6c3-48cc-b564-afac8ead5350" (UID: "223d0dff-a6c3-48cc-b564-afac8ead5350"). InnerVolumeSpecName "kube-api-access-ts7s6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:15:12.972413 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:12.972370 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-util" (OuterVolumeSpecName: "util") pod "223d0dff-a6c3-48cc-b564-afac8ead5350" (UID: "223d0dff-a6c3-48cc-b564-afac8ead5350"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:13.064759 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.064712 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tchg6\" (UniqueName: \"kubernetes.io/projected/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-kube-api-access-tchg6\") pod \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " Jul 10 19:15:13.064759 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.064765 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-util\") pod \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " Jul 10 19:15:13.065022 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.064834 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-bundle\") pod \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\" (UID: \"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f\") " Jul 10 19:15:13.065071 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.065058 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-util\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:13.065103 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.065073 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts7s6\" (UniqueName: \"kubernetes.io/projected/223d0dff-a6c3-48cc-b564-afac8ead5350-kube-api-access-ts7s6\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:13.065103 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.065082 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d0dff-a6c3-48cc-b564-afac8ead5350-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:13.065383 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.065355 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-bundle" (OuterVolumeSpecName: "bundle") pod "9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" (UID: "9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:13.067033 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.066983 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-kube-api-access-tchg6" (OuterVolumeSpecName: "kube-api-access-tchg6") pod "9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" (UID: "9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f"). InnerVolumeSpecName "kube-api-access-tchg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:15:13.070357 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.070332 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-util" (OuterVolumeSpecName: "util") pod "9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" (UID: "9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:13.165687 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.165638 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tchg6\" (UniqueName: \"kubernetes.io/projected/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-kube-api-access-tchg6\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:13.165687 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.165680 2556 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-util\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:13.165687 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.165689 2556 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:13.763492 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.763458 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" event={"ID":"9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f","Type":"ContainerDied","Data":"8f65eb69c76111786cede9cbe6ee5ca06ea12a00c22d77b943647f2fb3a4d7d5"} Jul 10 19:15:13.763852 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.763518 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f65eb69c76111786cede9cbe6ee5ca06ea12a00c22d77b943647f2fb3a4d7d5" Jul 10 19:15:13.763852 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.763466 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efrp8g9" Jul 10 19:15:13.765450 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.765426 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" event={"ID":"223d0dff-a6c3-48cc-b564-afac8ead5350","Type":"ContainerDied","Data":"dd09b4147a3f79b0d784ea43bbc588c7528e5f05c56833e678a536726e9924d9"} Jul 10 19:15:13.765560 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.765463 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd09b4147a3f79b0d784ea43bbc588c7528e5f05c56833e678a536726e9924d9" Jul 10 19:15:13.765560 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:13.765488 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff36j4vmq" Jul 10 19:15:24.269785 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.269737 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r"] Jul 10 19:15:24.270400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270294 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerName="util" Jul 10 19:15:24.270400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270316 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerName="util" Jul 10 19:15:24.270400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270343 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerName="util" Jul 10 19:15:24.270400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270351 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerName="util" Jul 10 19:15:24.270400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270366 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerName="pull" Jul 10 19:15:24.270400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270375 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerName="pull" Jul 10 19:15:24.270400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270383 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerName="extract" Jul 10 19:15:24.270400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270391 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerName="extract" Jul 10 19:15:24.270400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270400 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerName="pull" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270408 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerName="pull" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270421 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerName="extract" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270429 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerName="extract" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270444 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerName="util" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270452 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerName="util" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270463 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerName="extract" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270470 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerName="extract" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270479 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerName="pull" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270486 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerName="pull" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270499 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerName="util" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270506 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerName="util" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270514 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerName="pull" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270523 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerName="pull" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270531 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerName="extract" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270539 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerName="extract" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270630 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bbcf0f5-4989-4d02-9734-db81ee444a40" containerName="extract" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270642 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9140f2e3-f0dd-4cf8-aa8e-2aa89a403b5f" containerName="extract" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270653 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a4be61a-c40f-42eb-a7bb-cbfbb209424d" containerName="extract" Jul 10 19:15:24.270814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.270663 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="223d0dff-a6c3-48cc-b564-afac8ead5350" containerName="extract" Jul 10 19:15:24.274305 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.274272 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:24.277361 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.277338 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nzzvk\"" Jul 10 19:15:24.277361 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.277346 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 19:15:24.277528 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.277347 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 19:15:24.292299 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.292270 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r"] Jul 10 19:15:24.364474 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.364408 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a0aa562-7776-4da8-aa8d-4dea66732092-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" (UID: \"0a0aa562-7776-4da8-aa8d-4dea66732092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:24.364660 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.364495 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smrmv\" (UniqueName: \"kubernetes.io/projected/0a0aa562-7776-4da8-aa8d-4dea66732092-kube-api-access-smrmv\") pod \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" (UID: \"0a0aa562-7776-4da8-aa8d-4dea66732092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:24.465714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.465673 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a0aa562-7776-4da8-aa8d-4dea66732092-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" (UID: \"0a0aa562-7776-4da8-aa8d-4dea66732092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:24.465714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.465713 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smrmv\" (UniqueName: \"kubernetes.io/projected/0a0aa562-7776-4da8-aa8d-4dea66732092-kube-api-access-smrmv\") pod \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" (UID: \"0a0aa562-7776-4da8-aa8d-4dea66732092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:24.466187 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.466161 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a0aa562-7776-4da8-aa8d-4dea66732092-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" (UID: \"0a0aa562-7776-4da8-aa8d-4dea66732092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:24.475647 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.475618 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smrmv\" (UniqueName: \"kubernetes.io/projected/0a0aa562-7776-4da8-aa8d-4dea66732092-kube-api-access-smrmv\") pod \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" (UID: \"0a0aa562-7776-4da8-aa8d-4dea66732092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:24.585825 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.585734 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:24.753113 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.753084 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r"] Jul 10 19:15:24.759068 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:24.759035 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0aa562_7776_4da8_aa8d_4dea66732092.slice/crio-93d0627e39a4171bbe21e859873acf1a251d4535a7fa089b9a08ea9b61725ca2 WatchSource:0}: Error finding container 93d0627e39a4171bbe21e859873acf1a251d4535a7fa089b9a08ea9b61725ca2: Status 404 returned error can't find the container with id 93d0627e39a4171bbe21e859873acf1a251d4535a7fa089b9a08ea9b61725ca2 Jul 10 19:15:24.824621 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:24.824589 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" event={"ID":"0a0aa562-7776-4da8-aa8d-4dea66732092","Type":"ContainerStarted","Data":"93d0627e39a4171bbe21e859873acf1a251d4535a7fa089b9a08ea9b61725ca2"} Jul 10 19:15:27.278844 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:27.278810 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-558c6fb9c4-gtb4k"] Jul 10 19:15:27.282577 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:27.282549 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" Jul 10 19:15:27.287617 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:27.287391 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-br4qc\"" Jul 10 19:15:27.401084 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:27.400927 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvf7r\" (UniqueName: \"kubernetes.io/projected/a420d721-86ed-48b9-9eed-e69e2396256b-kube-api-access-zvf7r\") pod \"authorino-operator-558c6fb9c4-gtb4k\" (UID: \"a420d721-86ed-48b9-9eed-e69e2396256b\") " pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" Jul 10 19:15:27.402952 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:27.402849 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-558c6fb9c4-gtb4k"] Jul 10 19:15:27.502438 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:27.502403 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvf7r\" (UniqueName: \"kubernetes.io/projected/a420d721-86ed-48b9-9eed-e69e2396256b-kube-api-access-zvf7r\") pod \"authorino-operator-558c6fb9c4-gtb4k\" (UID: \"a420d721-86ed-48b9-9eed-e69e2396256b\") " pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" Jul 10 19:15:27.523083 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:27.523016 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvf7r\" (UniqueName: \"kubernetes.io/projected/a420d721-86ed-48b9-9eed-e69e2396256b-kube-api-access-zvf7r\") pod \"authorino-operator-558c6fb9c4-gtb4k\" (UID: \"a420d721-86ed-48b9-9eed-e69e2396256b\") " pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" Jul 10 19:15:27.599397 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:27.599293 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" Jul 10 19:15:28.830944 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:28.830920 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-558c6fb9c4-gtb4k"] Jul 10 19:15:28.833918 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:28.833819 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda420d721_86ed_48b9_9eed_e69e2396256b.slice/crio-c1b792d146bdfef244c266f822fb3fb327489921e6d6f46941d156ef6df80b6a WatchSource:0}: Error finding container c1b792d146bdfef244c266f822fb3fb327489921e6d6f46941d156ef6df80b6a: Status 404 returned error can't find the container with id c1b792d146bdfef244c266f822fb3fb327489921e6d6f46941d156ef6df80b6a Jul 10 19:15:28.859159 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:28.859125 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" event={"ID":"0a0aa562-7776-4da8-aa8d-4dea66732092","Type":"ContainerStarted","Data":"ddc114660d393eddd91e6271f64073b11fbcd032dd72b2aa281fe33c33815576"} Jul 10 19:15:28.859359 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:28.859186 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:28.860573 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:28.860549 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" event={"ID":"a420d721-86ed-48b9-9eed-e69e2396256b","Type":"ContainerStarted","Data":"c1b792d146bdfef244c266f822fb3fb327489921e6d6f46941d156ef6df80b6a"} Jul 10 19:15:28.884153 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:28.884099 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" podStartSLOduration=0.884691082 podStartE2EDuration="4.884079654s" totalImagesPullingTime="3.999388572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:15:28.881408094 +0000 UTC m=+480.897866466" watchObservedRunningTime="2026-07-10 19:15:28.884079654 +0000 UTC m=+480.900538024" Jul 10 19:15:29.088328 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.088295 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf"] Jul 10 19:15:29.091840 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.091817 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" Jul 10 19:15:29.094787 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.094766 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 10 19:15:29.094930 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.094796 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-k7vt2\"" Jul 10 19:15:29.106308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.106271 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf"] Jul 10 19:15:29.114352 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.114316 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz59f\" (UniqueName: \"kubernetes.io/projected/c3badc99-0410-4cf2-8044-19de0ae1be89-kube-api-access-bz59f\") pod \"dns-operator-controller-manager-5546559d7-d5rlf\" (UID: \"c3badc99-0410-4cf2-8044-19de0ae1be89\") " pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" Jul 10 19:15:29.215197 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.215159 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz59f\" (UniqueName: \"kubernetes.io/projected/c3badc99-0410-4cf2-8044-19de0ae1be89-kube-api-access-bz59f\") pod \"dns-operator-controller-manager-5546559d7-d5rlf\" (UID: \"c3badc99-0410-4cf2-8044-19de0ae1be89\") " pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" Jul 10 19:15:29.238016 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.237982 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz59f\" (UniqueName: \"kubernetes.io/projected/c3badc99-0410-4cf2-8044-19de0ae1be89-kube-api-access-bz59f\") pod \"dns-operator-controller-manager-5546559d7-d5rlf\" (UID: \"c3badc99-0410-4cf2-8044-19de0ae1be89\") " pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" Jul 10 19:15:29.404833 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.404721 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" Jul 10 19:15:29.575354 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.575323 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf"] Jul 10 19:15:29.578012 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:29.577969 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3badc99_0410_4cf2_8044_19de0ae1be89.slice/crio-ea49c47b9e99c723182a8a17f9ea080b2656b05a9be61329d227adb168f63e3d WatchSource:0}: Error finding container ea49c47b9e99c723182a8a17f9ea080b2656b05a9be61329d227adb168f63e3d: Status 404 returned error can't find the container with id ea49c47b9e99c723182a8a17f9ea080b2656b05a9be61329d227adb168f63e3d Jul 10 19:15:29.869572 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:29.869530 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" event={"ID":"c3badc99-0410-4cf2-8044-19de0ae1be89","Type":"ContainerStarted","Data":"ea49c47b9e99c723182a8a17f9ea080b2656b05a9be61329d227adb168f63e3d"} Jul 10 19:15:30.878421 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:30.878383 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" event={"ID":"a420d721-86ed-48b9-9eed-e69e2396256b","Type":"ContainerStarted","Data":"27dd3339eb5cbc77c17ef5308256500a32d246ed58ef14dea9bfda6f67b32e23"} Jul 10 19:15:30.878973 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:30.878533 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" Jul 10 19:15:30.900517 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:30.900450 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" podStartSLOduration=2.555544828 podStartE2EDuration="3.900429827s" totalImagesPullingTime="1.344884999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:15:30.898780251 +0000 UTC m=+482.915238622" watchObservedRunningTime="2026-07-10 19:15:30.900429827 +0000 UTC m=+482.916888227" Jul 10 19:15:31.511122 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:31.511075 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt"] Jul 10 19:15:31.520967 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:31.520935 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:31.524684 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:31.524651 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-75z4c\"" Jul 10 19:15:31.533180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:31.528335 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt"] Jul 10 19:15:31.536493 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:31.536439 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgxpp\" (UniqueName: \"kubernetes.io/projected/e149eaf2-6f89-40db-8247-6d61106c382d-kube-api-access-pgxpp\") pod \"limitador-operator-controller-manager-7d6bb9cf99-sc5pt\" (UID: \"e149eaf2-6f89-40db-8247-6d61106c382d\") " pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:31.637351 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:31.637305 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgxpp\" (UniqueName: \"kubernetes.io/projected/e149eaf2-6f89-40db-8247-6d61106c382d-kube-api-access-pgxpp\") pod \"limitador-operator-controller-manager-7d6bb9cf99-sc5pt\" (UID: \"e149eaf2-6f89-40db-8247-6d61106c382d\") " pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:31.649271 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:31.649237 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgxpp\" (UniqueName: \"kubernetes.io/projected/e149eaf2-6f89-40db-8247-6d61106c382d-kube-api-access-pgxpp\") pod \"limitador-operator-controller-manager-7d6bb9cf99-sc5pt\" (UID: \"e149eaf2-6f89-40db-8247-6d61106c382d\") " pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:31.844172 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:31.844087 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:32.338219 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:32.338193 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt"] Jul 10 19:15:32.341355 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:32.341319 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode149eaf2_6f89_40db_8247_6d61106c382d.slice/crio-3201aa2d25ac1ca536bf46d72437a8fa253ed0cfe3c3d4122ed782b967267037 WatchSource:0}: Error finding container 3201aa2d25ac1ca536bf46d72437a8fa253ed0cfe3c3d4122ed782b967267037: Status 404 returned error can't find the container with id 3201aa2d25ac1ca536bf46d72437a8fa253ed0cfe3c3d4122ed782b967267037 Jul 10 19:15:32.896247 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:32.896208 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" event={"ID":"c3badc99-0410-4cf2-8044-19de0ae1be89","Type":"ContainerStarted","Data":"2a8d295f6c297de5e7f0fe1b2cb9f733e95f3b11185f0a5af30a941740660727"} Jul 10 19:15:32.896459 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:32.896345 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" Jul 10 19:15:32.897639 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:32.897612 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" event={"ID":"e149eaf2-6f89-40db-8247-6d61106c382d","Type":"ContainerStarted","Data":"3201aa2d25ac1ca536bf46d72437a8fa253ed0cfe3c3d4122ed782b967267037"} Jul 10 19:15:32.918944 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:32.918865 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" podStartSLOduration=1.245294179 podStartE2EDuration="3.918846035s" totalImagesPullingTime="2.673551856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:15:32.915966649 +0000 UTC m=+484.932425020" watchObservedRunningTime="2026-07-10 19:15:32.918846035 +0000 UTC m=+484.935304405" Jul 10 19:15:34.911743 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:34.911704 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" event={"ID":"e149eaf2-6f89-40db-8247-6d61106c382d","Type":"ContainerStarted","Data":"0962a242edb2c6f67dc5810ba7b895bba6df7ccb4acb39bac0757f8ce3a4d8fd"} Jul 10 19:15:34.912184 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:34.911775 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:34.932965 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:34.932906 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" podStartSLOduration=2.225812433 podStartE2EDuration="3.932885973s" totalImagesPullingTime="1.70707354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:15:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:15:34.930263956 +0000 UTC m=+486.946722341" watchObservedRunningTime="2026-07-10 19:15:34.932885973 +0000 UTC m=+486.949344402" Jul 10 19:15:39.872939 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:39.872907 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:41.689775 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.689743 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r"] Jul 10 19:15:41.690216 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.689965 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" containerName="manager" containerID="cri-o://ddc114660d393eddd91e6271f64073b11fbcd032dd72b2aa281fe33c33815576" gracePeriod=2 Jul 10 19:15:41.695916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.695865 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r"] Jul 10 19:15:41.722145 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.722105 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt"] Jul 10 19:15:41.722390 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.722361 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" containerName="manager" containerID="cri-o://0962a242edb2c6f67dc5810ba7b895bba6df7ccb4acb39bac0757f8ce3a4d8fd" gracePeriod=2 Jul 10 19:15:41.724400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.724375 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:41.724951 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.724916 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:41.730779 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.730745 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752"] Jul 10 19:15:41.731523 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.731376 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" containerName="manager" Jul 10 19:15:41.731523 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.731402 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" containerName="manager" Jul 10 19:15:41.731702 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.731533 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" containerName="manager" Jul 10 19:15:41.734714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.734691 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:41.735995 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.735970 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt"] Jul 10 19:15:41.739793 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.739762 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:41.742709 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.742686 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-7d6bb9cf99-sc5pt\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:41.746050 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.746024 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-7d6bb9cf99-sc5pt\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:41.758354 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.758324 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p"] Jul 10 19:15:41.758714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.758700 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" containerName="manager" Jul 10 19:15:41.758714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.758715 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" containerName="manager" Jul 10 19:15:41.758905 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.758781 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" containerName="manager" Jul 10 19:15:41.762607 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.762572 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752"] Jul 10 19:15:41.762737 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.762710 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" Jul 10 19:15:41.785200 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.785153 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:41.787888 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.787847 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p"] Jul 10 19:15:41.788068 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.788044 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-7d6bb9cf99-sc5pt\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:41.812924 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.812862 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:41.822115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.822086 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602de9c-c7ce-4277-82d9-35395d6cb47f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6c647fc79-4f752\" (UID: \"2602de9c-c7ce-4277-82d9-35395d6cb47f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:41.822267 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.822119 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjg6g\" (UniqueName: \"kubernetes.io/projected/65ca1260-f825-4690-95f8-4663cf06f13c-kube-api-access-wjg6g\") pod \"limitador-operator-controller-manager-7d6bb9cf99-qrp4p\" (UID: \"65ca1260-f825-4690-95f8-4663cf06f13c\") " pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" Jul 10 19:15:41.822267 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.822147 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtqh2\" (UniqueName: \"kubernetes.io/projected/2602de9c-c7ce-4277-82d9-35395d6cb47f-kube-api-access-mtqh2\") pod \"kuadrant-operator-controller-manager-6c647fc79-4f752\" (UID: \"2602de9c-c7ce-4277-82d9-35395d6cb47f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:41.892411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.892304 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-558c6fb9c4-gtb4k" Jul 10 19:15:41.895301 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.895261 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:41.920597 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.920557 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-7d6bb9cf99-sc5pt\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:41.923065 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.923036 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602de9c-c7ce-4277-82d9-35395d6cb47f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6c647fc79-4f752\" (UID: \"2602de9c-c7ce-4277-82d9-35395d6cb47f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:41.923300 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.923280 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjg6g\" (UniqueName: \"kubernetes.io/projected/65ca1260-f825-4690-95f8-4663cf06f13c-kube-api-access-wjg6g\") pod \"limitador-operator-controller-manager-7d6bb9cf99-qrp4p\" (UID: \"65ca1260-f825-4690-95f8-4663cf06f13c\") " pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" Jul 10 19:15:41.923573 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.923544 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtqh2\" (UniqueName: \"kubernetes.io/projected/2602de9c-c7ce-4277-82d9-35395d6cb47f-kube-api-access-mtqh2\") pod \"kuadrant-operator-controller-manager-6c647fc79-4f752\" (UID: \"2602de9c-c7ce-4277-82d9-35395d6cb47f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:41.923721 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.923695 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602de9c-c7ce-4277-82d9-35395d6cb47f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6c647fc79-4f752\" (UID: \"2602de9c-c7ce-4277-82d9-35395d6cb47f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:41.936571 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.936535 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjg6g\" (UniqueName: \"kubernetes.io/projected/65ca1260-f825-4690-95f8-4663cf06f13c-kube-api-access-wjg6g\") pod \"limitador-operator-controller-manager-7d6bb9cf99-qrp4p\" (UID: \"65ca1260-f825-4690-95f8-4663cf06f13c\") " pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" Jul 10 19:15:41.937207 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.937144 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtqh2\" (UniqueName: \"kubernetes.io/projected/2602de9c-c7ce-4277-82d9-35395d6cb47f-kube-api-access-mtqh2\") pod \"kuadrant-operator-controller-manager-6c647fc79-4f752\" (UID: \"2602de9c-c7ce-4277-82d9-35395d6cb47f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:41.956986 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.956900 2556 generic.go:356] "Generic (PLEG): container finished" podID="e149eaf2-6f89-40db-8247-6d61106c382d" containerID="0962a242edb2c6f67dc5810ba7b895bba6df7ccb4acb39bac0757f8ce3a4d8fd" exitCode=0 Jul 10 19:15:41.958951 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.958928 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a0aa562-7776-4da8-aa8d-4dea66732092" containerID="ddc114660d393eddd91e6271f64073b11fbcd032dd72b2aa281fe33c33815576" exitCode=0 Jul 10 19:15:41.974836 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.974813 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:41.977318 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:41.977295 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:41.977422 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.977312 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-7d6bb9cf99-sc5pt\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:41.980107 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.980083 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:41.982535 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.982512 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-7d6bb9cf99-sc5pt\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:41.987675 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:41.987652 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6c647fc79-vl52r\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:42.025206 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.025175 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smrmv\" (UniqueName: \"kubernetes.io/projected/0a0aa562-7776-4da8-aa8d-4dea66732092-kube-api-access-smrmv\") pod \"0a0aa562-7776-4da8-aa8d-4dea66732092\" (UID: \"0a0aa562-7776-4da8-aa8d-4dea66732092\") " Jul 10 19:15:42.025360 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.025284 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a0aa562-7776-4da8-aa8d-4dea66732092-extensions-socket-volume\") pod \"0a0aa562-7776-4da8-aa8d-4dea66732092\" (UID: \"0a0aa562-7776-4da8-aa8d-4dea66732092\") " Jul 10 19:15:42.025360 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.025332 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgxpp\" (UniqueName: \"kubernetes.io/projected/e149eaf2-6f89-40db-8247-6d61106c382d-kube-api-access-pgxpp\") pod \"e149eaf2-6f89-40db-8247-6d61106c382d\" (UID: \"e149eaf2-6f89-40db-8247-6d61106c382d\") " Jul 10 19:15:42.025833 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.025786 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0aa562-7776-4da8-aa8d-4dea66732092-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0a0aa562-7776-4da8-aa8d-4dea66732092" (UID: "0a0aa562-7776-4da8-aa8d-4dea66732092"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:42.027655 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.027630 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e149eaf2-6f89-40db-8247-6d61106c382d-kube-api-access-pgxpp" (OuterVolumeSpecName: "kube-api-access-pgxpp") pod "e149eaf2-6f89-40db-8247-6d61106c382d" (UID: "e149eaf2-6f89-40db-8247-6d61106c382d"). InnerVolumeSpecName "kube-api-access-pgxpp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:15:42.027734 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.027653 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0aa562-7776-4da8-aa8d-4dea66732092-kube-api-access-smrmv" (OuterVolumeSpecName: "kube-api-access-smrmv") pod "0a0aa562-7776-4da8-aa8d-4dea66732092" (UID: "0a0aa562-7776-4da8-aa8d-4dea66732092"). InnerVolumeSpecName "kube-api-access-smrmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:15:42.126362 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.126325 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smrmv\" (UniqueName: \"kubernetes.io/projected/0a0aa562-7776-4da8-aa8d-4dea66732092-kube-api-access-smrmv\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:42.126362 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.126354 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a0aa562-7776-4da8-aa8d-4dea66732092-extensions-socket-volume\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:42.126362 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.126365 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgxpp\" (UniqueName: \"kubernetes.io/projected/e149eaf2-6f89-40db-8247-6d61106c382d-kube-api-access-pgxpp\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:42.132264 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.132237 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:42.139408 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.139375 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" Jul 10 19:15:42.313014 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.312982 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752"] Jul 10 19:15:42.325490 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:42.325462 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2602de9c_c7ce_4277_82d9_35395d6cb47f.slice/crio-825772a66f7c110616796a1e6ce8706caa78a84329353e0d8d095bcdb5da2854 WatchSource:0}: Error finding container 825772a66f7c110616796a1e6ce8706caa78a84329353e0d8d095bcdb5da2854: Status 404 returned error can't find the container with id 825772a66f7c110616796a1e6ce8706caa78a84329353e0d8d095bcdb5da2854 Jul 10 19:15:42.339630 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.339590 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p"] Jul 10 19:15:42.343335 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:42.343308 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65ca1260_f825_4690_95f8_4663cf06f13c.slice/crio-9e1a6c3f18aca125a042e861a57573ada91652394d847dfb19642faa75e0be0f WatchSource:0}: Error finding container 9e1a6c3f18aca125a042e861a57573ada91652394d847dfb19642faa75e0be0f: Status 404 returned error can't find the container with id 9e1a6c3f18aca125a042e861a57573ada91652394d847dfb19642faa75e0be0f Jul 10 19:15:42.519104 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.519070 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a0aa562-7776-4da8-aa8d-4dea66732092" path="/var/lib/kubelet/pods/0a0aa562-7776-4da8-aa8d-4dea66732092/volumes" Jul 10 19:15:42.519458 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.519441 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e149eaf2-6f89-40db-8247-6d61106c382d" path="/var/lib/kubelet/pods/e149eaf2-6f89-40db-8247-6d61106c382d/volumes" Jul 10 19:15:42.966757 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.966661 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" event={"ID":"2602de9c-c7ce-4277-82d9-35395d6cb47f","Type":"ContainerStarted","Data":"c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5"} Jul 10 19:15:42.966757 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.966720 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" event={"ID":"2602de9c-c7ce-4277-82d9-35395d6cb47f","Type":"ContainerStarted","Data":"825772a66f7c110616796a1e6ce8706caa78a84329353e0d8d095bcdb5da2854"} Jul 10 19:15:42.966757 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.966748 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:42.968207 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.968181 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-vl52r" Jul 10 19:15:42.968315 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.968244 2556 scope.go:117] "RemoveContainer" containerID="ddc114660d393eddd91e6271f64073b11fbcd032dd72b2aa281fe33c33815576" Jul 10 19:15:42.970117 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.969659 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-sc5pt" Jul 10 19:15:42.972351 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.972091 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" event={"ID":"65ca1260-f825-4690-95f8-4663cf06f13c","Type":"ContainerStarted","Data":"000fc52d8526422eabbe407d09a610e2ae450a4e606dc5edf8d5b3fd3ff411b2"} Jul 10 19:15:42.972351 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.972147 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" event={"ID":"65ca1260-f825-4690-95f8-4663cf06f13c","Type":"ContainerStarted","Data":"9e1a6c3f18aca125a042e861a57573ada91652394d847dfb19642faa75e0be0f"} Jul 10 19:15:42.972497 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.972439 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" Jul 10 19:15:42.979399 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.979383 2556 scope.go:117] "RemoveContainer" containerID="0962a242edb2c6f67dc5810ba7b895bba6df7ccb4acb39bac0757f8ce3a4d8fd" Jul 10 19:15:42.999634 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:42.999590 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" podStartSLOduration=1.999572987 podStartE2EDuration="1.999572987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:15:42.999203553 +0000 UTC m=+495.015661947" watchObservedRunningTime="2026-07-10 19:15:42.999572987 +0000 UTC m=+495.016031358" Jul 10 19:15:43.034269 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.034214 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" podStartSLOduration=2.034198011 podStartE2EDuration="2.034198011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:15:43.031633147 +0000 UTC m=+495.048091517" watchObservedRunningTime="2026-07-10 19:15:43.034198011 +0000 UTC m=+495.050656376" Jul 10 19:15:43.235795 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.235711 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk"] Jul 10 19:15:43.239347 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.239324 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:43.260624 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.260584 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk"] Jul 10 19:15:43.338748 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.338706 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-765mt\" (UniqueName: \"kubernetes.io/projected/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-kube-api-access-765mt\") pod \"kuadrant-operator-controller-manager-7569bc7c69-bk8wk\" (UID: \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:43.338956 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.338836 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7569bc7c69-bk8wk\" (UID: \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:43.439284 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.439234 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7569bc7c69-bk8wk\" (UID: \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:43.439284 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.439276 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-765mt\" (UniqueName: \"kubernetes.io/projected/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-kube-api-access-765mt\") pod \"kuadrant-operator-controller-manager-7569bc7c69-bk8wk\" (UID: \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:43.439663 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.439638 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7569bc7c69-bk8wk\" (UID: \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:43.453645 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.453612 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-765mt\" (UniqueName: \"kubernetes.io/projected/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-kube-api-access-765mt\") pod \"kuadrant-operator-controller-manager-7569bc7c69-bk8wk\" (UID: \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:43.549692 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.549653 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:43.712345 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.712317 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk"] Jul 10 19:15:43.715277 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:15:43.715252 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a765f6_b3ea_4fc7_ade3_9f10030c94f3.slice/crio-3f0b7e98b65be01272115dac8722cf90ab218e1a5f768568ca500eb5b00a5c09 WatchSource:0}: Error finding container 3f0b7e98b65be01272115dac8722cf90ab218e1a5f768568ca500eb5b00a5c09: Status 404 returned error can't find the container with id 3f0b7e98b65be01272115dac8722cf90ab218e1a5f768568ca500eb5b00a5c09 Jul 10 19:15:43.907077 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.906993 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5546559d7-d5rlf" Jul 10 19:15:43.957841 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.957805 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5775766f6c-5hgxn"] Jul 10 19:15:43.985501 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.985460 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" event={"ID":"12a765f6-b3ea-4fc7-ade3-9f10030c94f3","Type":"ContainerStarted","Data":"183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee"} Jul 10 19:15:43.985998 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.985517 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" event={"ID":"12a765f6-b3ea-4fc7-ade3-9f10030c94f3","Type":"ContainerStarted","Data":"3f0b7e98b65be01272115dac8722cf90ab218e1a5f768568ca500eb5b00a5c09"} Jul 10 19:15:43.985998 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:43.985568 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:44.021510 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:44.021453 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" podStartSLOduration=1.021430616 podStartE2EDuration="1.021430616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:15:44.019782224 +0000 UTC m=+496.036240596" watchObservedRunningTime="2026-07-10 19:15:44.021430616 +0000 UTC m=+496.037888986" Jul 10 19:15:53.989252 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:53.989217 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-7d6bb9cf99-qrp4p" Jul 10 19:15:53.989651 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:53.989383 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:54.996062 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:54.996028 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:15:55.174695 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:55.174656 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752"] Jul 10 19:15:55.174986 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:55.174959 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" podUID="2602de9c-c7ce-4277-82d9-35395d6cb47f" containerName="manager" containerID="cri-o://c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5" gracePeriod=10 Jul 10 19:15:55.440157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:55.440125 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:55.559842 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:55.559755 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602de9c-c7ce-4277-82d9-35395d6cb47f-extensions-socket-volume\") pod \"2602de9c-c7ce-4277-82d9-35395d6cb47f\" (UID: \"2602de9c-c7ce-4277-82d9-35395d6cb47f\") " Jul 10 19:15:55.560029 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:55.559905 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtqh2\" (UniqueName: \"kubernetes.io/projected/2602de9c-c7ce-4277-82d9-35395d6cb47f-kube-api-access-mtqh2\") pod \"2602de9c-c7ce-4277-82d9-35395d6cb47f\" (UID: \"2602de9c-c7ce-4277-82d9-35395d6cb47f\") " Jul 10 19:15:55.560203 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:55.560169 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2602de9c-c7ce-4277-82d9-35395d6cb47f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2602de9c-c7ce-4277-82d9-35395d6cb47f" (UID: "2602de9c-c7ce-4277-82d9-35395d6cb47f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:15:55.562229 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:55.562202 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2602de9c-c7ce-4277-82d9-35395d6cb47f-kube-api-access-mtqh2" (OuterVolumeSpecName: "kube-api-access-mtqh2") pod "2602de9c-c7ce-4277-82d9-35395d6cb47f" (UID: "2602de9c-c7ce-4277-82d9-35395d6cb47f"). InnerVolumeSpecName "kube-api-access-mtqh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:15:55.661451 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:55.661417 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtqh2\" (UniqueName: \"kubernetes.io/projected/2602de9c-c7ce-4277-82d9-35395d6cb47f-kube-api-access-mtqh2\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:55.661451 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:55.661446 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2602de9c-c7ce-4277-82d9-35395d6cb47f-extensions-socket-volume\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:15:56.062711 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.062672 2556 generic.go:356] "Generic (PLEG): container finished" podID="2602de9c-c7ce-4277-82d9-35395d6cb47f" containerID="c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5" exitCode=0 Jul 10 19:15:56.063204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.062717 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" Jul 10 19:15:56.063204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.062756 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" event={"ID":"2602de9c-c7ce-4277-82d9-35395d6cb47f","Type":"ContainerDied","Data":"c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5"} Jul 10 19:15:56.063204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.062815 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752" event={"ID":"2602de9c-c7ce-4277-82d9-35395d6cb47f","Type":"ContainerDied","Data":"825772a66f7c110616796a1e6ce8706caa78a84329353e0d8d095bcdb5da2854"} Jul 10 19:15:56.063204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.062844 2556 scope.go:117] "RemoveContainer" containerID="c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5" Jul 10 19:15:56.072941 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.072922 2556 scope.go:117] "RemoveContainer" containerID="c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5" Jul 10 19:15:56.073294 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:15:56.073274 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5\": container with ID starting with c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5 not found: ID does not exist" containerID="c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5" Jul 10 19:15:56.073345 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.073306 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5"} err="failed to get container status \"c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5\": rpc error: code = NotFound desc = could not find container \"c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5\": container with ID starting with c330c4dde9afb7609625aa44b1924f005fb7a576a3420e78e1d7c58f8ddde3d5 not found: ID does not exist" Jul 10 19:15:56.090681 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.090650 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752"] Jul 10 19:15:56.093419 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.093396 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6c647fc79-4f752"] Jul 10 19:15:56.517432 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:15:56.517395 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2602de9c-c7ce-4277-82d9-35395d6cb47f" path="/var/lib/kubelet/pods/2602de9c-c7ce-4277-82d9-35395d6cb47f/volumes" Jul 10 19:16:08.992160 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:08.992096 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5775766f6c-5hgxn" podUID="2bd56573-9764-47c7-92ce-09db536d1e9a" containerName="console" containerID="cri-o://10fe93426fd7ca0c94af4a3a37f19656749ab0b7ae2b005200a3787b6bafbbae" gracePeriod=15 Jul 10 19:16:09.140823 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.140790 2556 generic.go:356] "Generic (PLEG): container finished" podID="2bd56573-9764-47c7-92ce-09db536d1e9a" containerID="10fe93426fd7ca0c94af4a3a37f19656749ab0b7ae2b005200a3787b6bafbbae" exitCode=0 Jul 10 19:16:09.141052 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.140866 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5775766f6c-5hgxn" event={"ID":"2bd56573-9764-47c7-92ce-09db536d1e9a","Type":"ContainerDied","Data":"10fe93426fd7ca0c94af4a3a37f19656749ab0b7ae2b005200a3787b6bafbbae"} Jul 10 19:16:09.252015 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.251952 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:16:09.383223 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383189 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-trusted-ca-bundle\") pod \"2bd56573-9764-47c7-92ce-09db536d1e9a\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " Jul 10 19:16:09.383223 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383228 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87ljf\" (UniqueName: \"kubernetes.io/projected/2bd56573-9764-47c7-92ce-09db536d1e9a-kube-api-access-87ljf\") pod \"2bd56573-9764-47c7-92ce-09db536d1e9a\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " Jul 10 19:16:09.383465 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383259 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-serving-cert\") pod \"2bd56573-9764-47c7-92ce-09db536d1e9a\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " Jul 10 19:16:09.383465 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383280 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-oauth-config\") pod \"2bd56573-9764-47c7-92ce-09db536d1e9a\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " Jul 10 19:16:09.383465 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383404 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-console-config\") pod \"2bd56573-9764-47c7-92ce-09db536d1e9a\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " Jul 10 19:16:09.383632 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383466 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-oauth-serving-cert\") pod \"2bd56573-9764-47c7-92ce-09db536d1e9a\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " Jul 10 19:16:09.383632 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383494 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-service-ca\") pod \"2bd56573-9764-47c7-92ce-09db536d1e9a\" (UID: \"2bd56573-9764-47c7-92ce-09db536d1e9a\") " Jul 10 19:16:09.383799 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383759 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2bd56573-9764-47c7-92ce-09db536d1e9a" (UID: "2bd56573-9764-47c7-92ce-09db536d1e9a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:16:09.383864 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383839 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2bd56573-9764-47c7-92ce-09db536d1e9a" (UID: "2bd56573-9764-47c7-92ce-09db536d1e9a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:16:09.383956 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.383847 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-console-config" (OuterVolumeSpecName: "console-config") pod "2bd56573-9764-47c7-92ce-09db536d1e9a" (UID: "2bd56573-9764-47c7-92ce-09db536d1e9a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:16:09.384156 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.384133 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-service-ca" (OuterVolumeSpecName: "service-ca") pod "2bd56573-9764-47c7-92ce-09db536d1e9a" (UID: "2bd56573-9764-47c7-92ce-09db536d1e9a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 19:16:09.385622 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.385599 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2bd56573-9764-47c7-92ce-09db536d1e9a" (UID: "2bd56573-9764-47c7-92ce-09db536d1e9a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:16:09.385622 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.385597 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd56573-9764-47c7-92ce-09db536d1e9a-kube-api-access-87ljf" (OuterVolumeSpecName: "kube-api-access-87ljf") pod "2bd56573-9764-47c7-92ce-09db536d1e9a" (UID: "2bd56573-9764-47c7-92ce-09db536d1e9a"). InnerVolumeSpecName "kube-api-access-87ljf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:16:09.385738 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.385624 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2bd56573-9764-47c7-92ce-09db536d1e9a" (UID: "2bd56573-9764-47c7-92ce-09db536d1e9a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:16:09.485030 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.484989 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87ljf\" (UniqueName: \"kubernetes.io/projected/2bd56573-9764-47c7-92ce-09db536d1e9a-kube-api-access-87ljf\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:09.485030 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.485023 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-serving-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:09.485030 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.485037 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bd56573-9764-47c7-92ce-09db536d1e9a-console-oauth-config\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:09.485325 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.485052 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-console-config\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:09.485325 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.485063 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-oauth-serving-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:09.485325 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.485075 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-service-ca\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:09.485325 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:09.485086 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bd56573-9764-47c7-92ce-09db536d1e9a-trusted-ca-bundle\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:10.148308 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:10.148276 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5775766f6c-5hgxn" Jul 10 19:16:10.148791 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:10.148278 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5775766f6c-5hgxn" event={"ID":"2bd56573-9764-47c7-92ce-09db536d1e9a","Type":"ContainerDied","Data":"508cfdddb897376628c6916de181da36f183015dc9be1019867d92c4e967538d"} Jul 10 19:16:10.148791 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:10.148409 2556 scope.go:117] "RemoveContainer" containerID="10fe93426fd7ca0c94af4a3a37f19656749ab0b7ae2b005200a3787b6bafbbae" Jul 10 19:16:10.171820 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:10.171781 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5775766f6c-5hgxn"] Jul 10 19:16:10.177998 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:10.177970 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5775766f6c-5hgxn"] Jul 10 19:16:10.518231 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:10.518195 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bd56573-9764-47c7-92ce-09db536d1e9a" path="/var/lib/kubelet/pods/2bd56573-9764-47c7-92ce-09db536d1e9a/volumes" Jul 10 19:16:11.640450 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.640416 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n"] Jul 10 19:16:11.641018 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.641000 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2602de9c-c7ce-4277-82d9-35395d6cb47f" containerName="manager" Jul 10 19:16:11.641115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.641020 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2602de9c-c7ce-4277-82d9-35395d6cb47f" containerName="manager" Jul 10 19:16:11.641115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.641034 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bd56573-9764-47c7-92ce-09db536d1e9a" containerName="console" Jul 10 19:16:11.641115 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.641042 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bd56573-9764-47c7-92ce-09db536d1e9a" containerName="console" Jul 10 19:16:11.641262 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.641124 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2602de9c-c7ce-4277-82d9-35395d6cb47f" containerName="manager" Jul 10 19:16:11.641262 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.641139 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bd56573-9764-47c7-92ce-09db536d1e9a" containerName="console" Jul 10 19:16:11.644091 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.644066 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.646771 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.646746 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-h9zcz\"" Jul 10 19:16:11.659146 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.659123 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n"] Jul 10 19:16:11.806395 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.806353 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/821b101f-507a-422a-83c3-58207b76e691-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.806395 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.806390 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/821b101f-507a-422a-83c3-58207b76e691-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.806654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.806463 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.806654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.806517 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.806654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.806545 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.806654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.806575 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.806654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.806607 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/821b101f-507a-422a-83c3-58207b76e691-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.806654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.806622 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpr5f\" (UniqueName: \"kubernetes.io/projected/821b101f-507a-422a-83c3-58207b76e691-kube-api-access-wpr5f\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.806654 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.806640 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.907805 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.907706 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/821b101f-507a-422a-83c3-58207b76e691-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.907805 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.907741 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/821b101f-507a-422a-83c3-58207b76e691-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.907805 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.907772 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.907805 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.907796 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908193 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.907818 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908193 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.907862 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908193 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.907934 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/821b101f-507a-422a-83c3-58207b76e691-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908193 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.907960 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpr5f\" (UniqueName: \"kubernetes.io/projected/821b101f-507a-422a-83c3-58207b76e691-kube-api-access-wpr5f\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908193 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.907989 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908447 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.908429 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908503 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.908479 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908557 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.908502 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908635 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.908608 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.908808 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.908788 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/821b101f-507a-422a-83c3-58207b76e691-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.910542 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.910521 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/821b101f-507a-422a-83c3-58207b76e691-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.910776 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.910756 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/821b101f-507a-422a-83c3-58207b76e691-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.917975 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.917945 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/821b101f-507a-422a-83c3-58207b76e691-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.918611 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.918586 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpr5f\" (UniqueName: \"kubernetes.io/projected/821b101f-507a-422a-83c3-58207b76e691-kube-api-access-wpr5f\") pod \"maas-default-gateway-openshift-default-56654dcf7-sc64n\" (UID: \"821b101f-507a-422a-83c3-58207b76e691\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:11.955418 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:11.955384 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:12.113483 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:12.113451 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n"] Jul 10 19:16:12.115178 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:16:12.115151 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821b101f_507a_422a_83c3_58207b76e691.slice/crio-c47a507633bc63818abb345262e6d8f7e5f17a5503613f24d8534b9dac3fd8ca WatchSource:0}: Error finding container c47a507633bc63818abb345262e6d8f7e5f17a5503613f24d8534b9dac3fd8ca: Status 404 returned error can't find the container with id c47a507633bc63818abb345262e6d8f7e5f17a5503613f24d8534b9dac3fd8ca Jul 10 19:16:12.117710 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:12.117676 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:16:12.117808 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:12.117761 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:16:12.117900 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:12.117805 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:16:12.163372 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:12.163345 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" event={"ID":"821b101f-507a-422a-83c3-58207b76e691","Type":"ContainerStarted","Data":"c47a507633bc63818abb345262e6d8f7e5f17a5503613f24d8534b9dac3fd8ca"} Jul 10 19:16:13.172363 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:13.172327 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" event={"ID":"821b101f-507a-422a-83c3-58207b76e691","Type":"ContainerStarted","Data":"e6048b826098e7af6827794fc5e24af371aded2cd13cec3ce525d9de97797c27"} Jul 10 19:16:13.198275 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:13.198226 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" podStartSLOduration=2.19820956 podStartE2EDuration="2.19820956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:16:13.197698914 +0000 UTC m=+525.214157297" watchObservedRunningTime="2026-07-10 19:16:13.19820956 +0000 UTC m=+525.214667930" Jul 10 19:16:13.956092 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:13.956047 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:13.961410 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:13.961380 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:14.179218 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:14.179183 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:14.180356 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:14.180335 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:16:15.220124 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.220088 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b6c89bb85-w2cdw"] Jul 10 19:16:15.223214 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.223196 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" Jul 10 19:16:15.227391 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.227368 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rkk8q\"" Jul 10 19:16:15.245737 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.245710 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-w2cdw"] Jul 10 19:16:15.342905 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.342838 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv26f\" (UniqueName: \"kubernetes.io/projected/8c692d75-31b6-4a7b-b5b7-0d2f3f184710-kube-api-access-pv26f\") pod \"authorino-b6c89bb85-w2cdw\" (UID: \"8c692d75-31b6-4a7b-b5b7-0d2f3f184710\") " pod="kuadrant-system/authorino-b6c89bb85-w2cdw" Jul 10 19:16:15.444250 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.444208 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv26f\" (UniqueName: \"kubernetes.io/projected/8c692d75-31b6-4a7b-b5b7-0d2f3f184710-kube-api-access-pv26f\") pod \"authorino-b6c89bb85-w2cdw\" (UID: \"8c692d75-31b6-4a7b-b5b7-0d2f3f184710\") " pod="kuadrant-system/authorino-b6c89bb85-w2cdw" Jul 10 19:16:15.462959 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.462927 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv26f\" (UniqueName: \"kubernetes.io/projected/8c692d75-31b6-4a7b-b5b7-0d2f3f184710-kube-api-access-pv26f\") pod \"authorino-b6c89bb85-w2cdw\" (UID: \"8c692d75-31b6-4a7b-b5b7-0d2f3f184710\") " pod="kuadrant-system/authorino-b6c89bb85-w2cdw" Jul 10 19:16:15.533263 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.533229 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" Jul 10 19:16:15.574836 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.574788 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8df697c9f-lmhgn"] Jul 10 19:16:15.577968 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.577942 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8df697c9f-lmhgn" Jul 10 19:16:15.601227 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.601192 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8df697c9f-lmhgn"] Jul 10 19:16:15.697942 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.697864 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-w2cdw"] Jul 10 19:16:15.700200 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:16:15.700160 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c692d75_31b6_4a7b_b5b7_0d2f3f184710.slice/crio-092e1f112bdfe515f6d6fdce5bbd136279abf3d8c155ec01e92f17b6270a23ed WatchSource:0}: Error finding container 092e1f112bdfe515f6d6fdce5bbd136279abf3d8c155ec01e92f17b6270a23ed: Status 404 returned error can't find the container with id 092e1f112bdfe515f6d6fdce5bbd136279abf3d8c155ec01e92f17b6270a23ed Jul 10 19:16:15.747686 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.747653 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hp7g\" (UniqueName: \"kubernetes.io/projected/cb053c11-b5a2-440a-a52b-4ddcc3dc8459-kube-api-access-2hp7g\") pod \"authorino-8df697c9f-lmhgn\" (UID: \"cb053c11-b5a2-440a-a52b-4ddcc3dc8459\") " pod="kuadrant-system/authorino-8df697c9f-lmhgn" Jul 10 19:16:15.849021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.848929 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hp7g\" (UniqueName: \"kubernetes.io/projected/cb053c11-b5a2-440a-a52b-4ddcc3dc8459-kube-api-access-2hp7g\") pod \"authorino-8df697c9f-lmhgn\" (UID: \"cb053c11-b5a2-440a-a52b-4ddcc3dc8459\") " pod="kuadrant-system/authorino-8df697c9f-lmhgn" Jul 10 19:16:15.861487 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.861446 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hp7g\" (UniqueName: \"kubernetes.io/projected/cb053c11-b5a2-440a-a52b-4ddcc3dc8459-kube-api-access-2hp7g\") pod \"authorino-8df697c9f-lmhgn\" (UID: \"cb053c11-b5a2-440a-a52b-4ddcc3dc8459\") " pod="kuadrant-system/authorino-8df697c9f-lmhgn" Jul 10 19:16:15.890122 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:15.890083 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8df697c9f-lmhgn" Jul 10 19:16:16.037448 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:16.037418 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8df697c9f-lmhgn"] Jul 10 19:16:16.038820 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:16:16.038788 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb053c11_b5a2_440a_a52b_4ddcc3dc8459.slice/crio-37b4447c788feef1d161de865103ac0171689dfee5cb30701307c74e0a6c4333 WatchSource:0}: Error finding container 37b4447c788feef1d161de865103ac0171689dfee5cb30701307c74e0a6c4333: Status 404 returned error can't find the container with id 37b4447c788feef1d161de865103ac0171689dfee5cb30701307c74e0a6c4333 Jul 10 19:16:16.192240 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:16.192148 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8df697c9f-lmhgn" event={"ID":"cb053c11-b5a2-440a-a52b-4ddcc3dc8459","Type":"ContainerStarted","Data":"37b4447c788feef1d161de865103ac0171689dfee5cb30701307c74e0a6c4333"} Jul 10 19:16:16.193428 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:16.193399 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" event={"ID":"8c692d75-31b6-4a7b-b5b7-0d2f3f184710","Type":"ContainerStarted","Data":"092e1f112bdfe515f6d6fdce5bbd136279abf3d8c155ec01e92f17b6270a23ed"} Jul 10 19:16:20.222806 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:20.222766 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8df697c9f-lmhgn" event={"ID":"cb053c11-b5a2-440a-a52b-4ddcc3dc8459","Type":"ContainerStarted","Data":"ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220"} Jul 10 19:16:20.238016 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:20.237962 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8df697c9f-lmhgn" podStartSLOduration=1.610028043 podStartE2EDuration="5.237944036s" totalImagesPullingTime="3.627915993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:16:20.236591402 +0000 UTC m=+532.253049775" watchObservedRunningTime="2026-07-10 19:16:20.237944036 +0000 UTC m=+532.254402406" Jul 10 19:16:20.264449 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:20.264409 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-w2cdw"] Jul 10 19:16:24.246617 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:24.246574 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" event={"ID":"8c692d75-31b6-4a7b-b5b7-0d2f3f184710","Type":"ContainerStarted","Data":"90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635"} Jul 10 19:16:24.247096 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:24.246636 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" podUID="8c692d75-31b6-4a7b-b5b7-0d2f3f184710" containerName="authorino" containerID="cri-o://90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635" gracePeriod=30 Jul 10 19:16:24.263306 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:24.263256 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" podStartSLOduration=1.6105093419999998 podStartE2EDuration="9.263236856s" totalImagesPullingTime="7.652727514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:16:24.260030645 +0000 UTC m=+536.276489017" watchObservedRunningTime="2026-07-10 19:16:24.263236856 +0000 UTC m=+536.279695264" Jul 10 19:16:24.496201 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:24.496168 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" Jul 10 19:16:24.528152 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:24.528043 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv26f\" (UniqueName: \"kubernetes.io/projected/8c692d75-31b6-4a7b-b5b7-0d2f3f184710-kube-api-access-pv26f\") pod \"8c692d75-31b6-4a7b-b5b7-0d2f3f184710\" (UID: \"8c692d75-31b6-4a7b-b5b7-0d2f3f184710\") " Jul 10 19:16:24.530344 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:24.530311 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c692d75-31b6-4a7b-b5b7-0d2f3f184710-kube-api-access-pv26f" (OuterVolumeSpecName: "kube-api-access-pv26f") pod "8c692d75-31b6-4a7b-b5b7-0d2f3f184710" (UID: "8c692d75-31b6-4a7b-b5b7-0d2f3f184710"). InnerVolumeSpecName "kube-api-access-pv26f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:16:24.628835 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:24.628793 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv26f\" (UniqueName: \"kubernetes.io/projected/8c692d75-31b6-4a7b-b5b7-0d2f3f184710-kube-api-access-pv26f\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:25.254064 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:25.254030 2556 generic.go:356] "Generic (PLEG): container finished" podID="8c692d75-31b6-4a7b-b5b7-0d2f3f184710" containerID="90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635" exitCode=0 Jul 10 19:16:25.254467 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:25.254082 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" Jul 10 19:16:25.254467 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:25.254121 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" event={"ID":"8c692d75-31b6-4a7b-b5b7-0d2f3f184710","Type":"ContainerDied","Data":"90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635"} Jul 10 19:16:25.254467 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:25.254169 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c89bb85-w2cdw" event={"ID":"8c692d75-31b6-4a7b-b5b7-0d2f3f184710","Type":"ContainerDied","Data":"092e1f112bdfe515f6d6fdce5bbd136279abf3d8c155ec01e92f17b6270a23ed"} Jul 10 19:16:25.254467 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:25.254191 2556 scope.go:117] "RemoveContainer" containerID="90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635" Jul 10 19:16:25.264325 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:25.264305 2556 scope.go:117] "RemoveContainer" containerID="90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635" Jul 10 19:16:25.264639 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:16:25.264620 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635\": container with ID starting with 90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635 not found: ID does not exist" containerID="90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635" Jul 10 19:16:25.264714 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:25.264648 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635"} err="failed to get container status \"90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635\": rpc error: code = NotFound desc = could not find container \"90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635\": container with ID starting with 90ea657923289945f4adfc2e4c73fc482e8fa3f1c8bf01b5a86a0a767a5b2635 not found: ID does not exist" Jul 10 19:16:25.276429 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:25.276399 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-w2cdw"] Jul 10 19:16:25.280163 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:25.280134 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b6c89bb85-w2cdw"] Jul 10 19:16:26.517326 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:26.517287 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c692d75-31b6-4a7b-b5b7-0d2f3f184710" path="/var/lib/kubelet/pods/8c692d75-31b6-4a7b-b5b7-0d2f3f184710/volumes" Jul 10 19:16:48.711383 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.711346 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dbc6c7c8d-hrxx9"] Jul 10 19:16:48.711936 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.711908 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c692d75-31b6-4a7b-b5b7-0d2f3f184710" containerName="authorino" Jul 10 19:16:48.711936 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.711926 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c692d75-31b6-4a7b-b5b7-0d2f3f184710" containerName="authorino" Jul 10 19:16:48.712100 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.712029 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c692d75-31b6-4a7b-b5b7-0d2f3f184710" containerName="authorino" Jul 10 19:16:48.717086 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.717062 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbc6c7c8d-hrxx9" Jul 10 19:16:48.722245 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.722211 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dbc6c7c8d-hrxx9"] Jul 10 19:16:48.795640 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.795601 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dbc6c7c8d-hrxx9"] Jul 10 19:16:48.795938 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:16:48.795916 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hmd85], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6dbc6c7c8d-hrxx9" podUID="2eaebb93-6b96-4582-afd2-f6ead495e577" Jul 10 19:16:48.826892 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.826228 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76dd94b454-xg755"] Jul 10 19:16:48.831737 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.831707 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:48.833561 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.833532 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76dd94b454-xg755"] Jul 10 19:16:48.834141 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.834120 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 19:16:48.845496 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.845461 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmd85\" (UniqueName: \"kubernetes.io/projected/2eaebb93-6b96-4582-afd2-f6ead495e577-kube-api-access-hmd85\") pod \"authorino-6dbc6c7c8d-hrxx9\" (UID: \"2eaebb93-6b96-4582-afd2-f6ead495e577\") " pod="kuadrant-system/authorino-6dbc6c7c8d-hrxx9" Jul 10 19:16:48.946034 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.945992 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdl5l\" (UniqueName: \"kubernetes.io/projected/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-kube-api-access-pdl5l\") pod \"authorino-76dd94b454-xg755\" (UID: \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\") " pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:48.946320 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.946294 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmd85\" (UniqueName: \"kubernetes.io/projected/2eaebb93-6b96-4582-afd2-f6ead495e577-kube-api-access-hmd85\") pod \"authorino-6dbc6c7c8d-hrxx9\" (UID: \"2eaebb93-6b96-4582-afd2-f6ead495e577\") " pod="kuadrant-system/authorino-6dbc6c7c8d-hrxx9" Jul 10 19:16:48.946485 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.946448 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-tls-cert\") pod \"authorino-76dd94b454-xg755\" (UID: \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\") " pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:48.948833 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.948814 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76dd94b454-xg755"] Jul 10 19:16:48.949089 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:16:48.949068 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pdl5l tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76dd94b454-xg755" podUID="d5de6ffe-a001-4c0f-a11e-614ddbbd76e0" Jul 10 19:16:48.959857 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.959826 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmd85\" (UniqueName: \"kubernetes.io/projected/2eaebb93-6b96-4582-afd2-f6ead495e577-kube-api-access-hmd85\") pod \"authorino-6dbc6c7c8d-hrxx9\" (UID: \"2eaebb93-6b96-4582-afd2-f6ead495e577\") " pod="kuadrant-system/authorino-6dbc6c7c8d-hrxx9" Jul 10 19:16:48.978566 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.978467 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-655655858d-pqpz7"] Jul 10 19:16:48.982655 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.982624 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:16:48.995445 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:48.995417 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-655655858d-pqpz7"] Jul 10 19:16:49.047370 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.047327 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-tls-cert\") pod \"authorino-76dd94b454-xg755\" (UID: \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\") " pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:49.047540 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.047427 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdl5l\" (UniqueName: \"kubernetes.io/projected/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-kube-api-access-pdl5l\") pod \"authorino-76dd94b454-xg755\" (UID: \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\") " pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:49.050048 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.050025 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-tls-cert\") pod \"authorino-76dd94b454-xg755\" (UID: \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\") " pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:49.055068 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.055043 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdl5l\" (UniqueName: \"kubernetes.io/projected/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-kube-api-access-pdl5l\") pod \"authorino-76dd94b454-xg755\" (UID: \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\") " pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:49.148411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.148371 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgd9\" (UniqueName: \"kubernetes.io/projected/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-kube-api-access-xwgd9\") pod \"authorino-655655858d-pqpz7\" (UID: \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\") " pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:16:49.148596 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.148430 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-tls-cert\") pod \"authorino-655655858d-pqpz7\" (UID: \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\") " pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:16:49.249818 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.249783 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgd9\" (UniqueName: \"kubernetes.io/projected/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-kube-api-access-xwgd9\") pod \"authorino-655655858d-pqpz7\" (UID: \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\") " pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:16:49.250020 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.249837 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-tls-cert\") pod \"authorino-655655858d-pqpz7\" (UID: \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\") " pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:16:49.252488 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.252457 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-tls-cert\") pod \"authorino-655655858d-pqpz7\" (UID: \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\") " pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:16:49.258415 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.258389 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgd9\" (UniqueName: \"kubernetes.io/projected/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-kube-api-access-xwgd9\") pod \"authorino-655655858d-pqpz7\" (UID: \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\") " pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:16:49.292235 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.292194 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:16:49.388698 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.388668 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:49.388861 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.388668 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbc6c7c8d-hrxx9" Jul 10 19:16:49.398768 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.398627 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:49.402697 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.402198 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbc6c7c8d-hrxx9" Jul 10 19:16:49.445174 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.445147 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-655655858d-pqpz7"] Jul 10 19:16:49.447513 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:16:49.447488 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f0e2c75_cf1a_46df_b681_ffabbd9261cf.slice/crio-f415167a8efac971fc7465cee67079c7b72229cbd4f221b6b52a5a7c4619c4dc WatchSource:0}: Error finding container f415167a8efac971fc7465cee67079c7b72229cbd4f221b6b52a5a7c4619c4dc: Status 404 returned error can't find the container with id f415167a8efac971fc7465cee67079c7b72229cbd4f221b6b52a5a7c4619c4dc Jul 10 19:16:49.552930 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.552819 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmd85\" (UniqueName: \"kubernetes.io/projected/2eaebb93-6b96-4582-afd2-f6ead495e577-kube-api-access-hmd85\") pod \"2eaebb93-6b96-4582-afd2-f6ead495e577\" (UID: \"2eaebb93-6b96-4582-afd2-f6ead495e577\") " Jul 10 19:16:49.552930 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.552865 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-tls-cert\") pod \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\" (UID: \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\") " Jul 10 19:16:49.553196 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.552969 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdl5l\" (UniqueName: \"kubernetes.io/projected/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-kube-api-access-pdl5l\") pod \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\" (UID: \"d5de6ffe-a001-4c0f-a11e-614ddbbd76e0\") " Jul 10 19:16:49.555218 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.555177 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eaebb93-6b96-4582-afd2-f6ead495e577-kube-api-access-hmd85" (OuterVolumeSpecName: "kube-api-access-hmd85") pod "2eaebb93-6b96-4582-afd2-f6ead495e577" (UID: "2eaebb93-6b96-4582-afd2-f6ead495e577"). InnerVolumeSpecName "kube-api-access-hmd85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:16:49.555333 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.555246 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d5de6ffe-a001-4c0f-a11e-614ddbbd76e0" (UID: "d5de6ffe-a001-4c0f-a11e-614ddbbd76e0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:16:49.555333 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.555255 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-kube-api-access-pdl5l" (OuterVolumeSpecName: "kube-api-access-pdl5l") pod "d5de6ffe-a001-4c0f-a11e-614ddbbd76e0" (UID: "d5de6ffe-a001-4c0f-a11e-614ddbbd76e0"). InnerVolumeSpecName "kube-api-access-pdl5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:16:49.654085 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.654040 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdl5l\" (UniqueName: \"kubernetes.io/projected/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-kube-api-access-pdl5l\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:49.654085 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.654077 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmd85\" (UniqueName: \"kubernetes.io/projected/2eaebb93-6b96-4582-afd2-f6ead495e577-kube-api-access-hmd85\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:49.654085 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:49.654090 2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0-tls-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:50.396146 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.396110 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbc6c7c8d-hrxx9" Jul 10 19:16:50.396146 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.396121 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-655655858d-pqpz7" event={"ID":"0f0e2c75-cf1a-46df-b681-ffabbd9261cf","Type":"ContainerStarted","Data":"0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa"} Jul 10 19:16:50.396146 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.396147 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76dd94b454-xg755" Jul 10 19:16:50.396597 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.396167 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-655655858d-pqpz7" event={"ID":"0f0e2c75-cf1a-46df-b681-ffabbd9261cf","Type":"ContainerStarted","Data":"f415167a8efac971fc7465cee67079c7b72229cbd4f221b6b52a5a7c4619c4dc"} Jul 10 19:16:50.413596 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.413544 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-655655858d-pqpz7" podStartSLOduration=1.9714428320000001 podStartE2EDuration="2.413528372s" totalImagesPullingTime="442.08554ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:16:50.410514012 +0000 UTC m=+562.426972384" watchObservedRunningTime="2026-07-10 19:16:50.413528372 +0000 UTC m=+562.429986743" Jul 10 19:16:50.437526 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.437485 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76dd94b454-xg755"] Jul 10 19:16:50.444674 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.444626 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8df697c9f-lmhgn"] Jul 10 19:16:50.444865 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.444821 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-8df697c9f-lmhgn" podUID="cb053c11-b5a2-440a-a52b-4ddcc3dc8459" containerName="authorino" containerID="cri-o://ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220" gracePeriod=30 Jul 10 19:16:50.447388 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.447361 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76dd94b454-xg755"] Jul 10 19:16:50.468710 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.468676 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dbc6c7c8d-hrxx9"] Jul 10 19:16:50.472447 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.472417 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dbc6c7c8d-hrxx9"] Jul 10 19:16:50.517371 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.517341 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eaebb93-6b96-4582-afd2-f6ead495e577" path="/var/lib/kubelet/pods/2eaebb93-6b96-4582-afd2-f6ead495e577/volumes" Jul 10 19:16:50.517603 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.517590 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5de6ffe-a001-4c0f-a11e-614ddbbd76e0" path="/var/lib/kubelet/pods/d5de6ffe-a001-4c0f-a11e-614ddbbd76e0/volumes" Jul 10 19:16:50.702805 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.702778 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8df697c9f-lmhgn" Jul 10 19:16:50.867168 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.867134 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hp7g\" (UniqueName: \"kubernetes.io/projected/cb053c11-b5a2-440a-a52b-4ddcc3dc8459-kube-api-access-2hp7g\") pod \"cb053c11-b5a2-440a-a52b-4ddcc3dc8459\" (UID: \"cb053c11-b5a2-440a-a52b-4ddcc3dc8459\") " Jul 10 19:16:50.869434 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.869407 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb053c11-b5a2-440a-a52b-4ddcc3dc8459-kube-api-access-2hp7g" (OuterVolumeSpecName: "kube-api-access-2hp7g") pod "cb053c11-b5a2-440a-a52b-4ddcc3dc8459" (UID: "cb053c11-b5a2-440a-a52b-4ddcc3dc8459"). InnerVolumeSpecName "kube-api-access-2hp7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:16:50.968054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:50.967967 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hp7g\" (UniqueName: \"kubernetes.io/projected/cb053c11-b5a2-440a-a52b-4ddcc3dc8459-kube-api-access-2hp7g\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:16:51.402794 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:51.402753 2556 generic.go:356] "Generic (PLEG): container finished" podID="cb053c11-b5a2-440a-a52b-4ddcc3dc8459" containerID="ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220" exitCode=0 Jul 10 19:16:51.402794 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:51.402798 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8df697c9f-lmhgn" Jul 10 19:16:51.403283 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:51.402848 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8df697c9f-lmhgn" event={"ID":"cb053c11-b5a2-440a-a52b-4ddcc3dc8459","Type":"ContainerDied","Data":"ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220"} Jul 10 19:16:51.403283 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:51.402925 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8df697c9f-lmhgn" event={"ID":"cb053c11-b5a2-440a-a52b-4ddcc3dc8459","Type":"ContainerDied","Data":"37b4447c788feef1d161de865103ac0171689dfee5cb30701307c74e0a6c4333"} Jul 10 19:16:51.403283 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:51.402947 2556 scope.go:117] "RemoveContainer" containerID="ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220" Jul 10 19:16:51.412859 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:51.412840 2556 scope.go:117] "RemoveContainer" containerID="ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220" Jul 10 19:16:51.413214 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:16:51.413194 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220\": container with ID starting with ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220 not found: ID does not exist" containerID="ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220" Jul 10 19:16:51.413266 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:51.413224 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220"} err="failed to get container status \"ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220\": rpc error: code = NotFound desc = could not find container \"ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220\": container with ID starting with ffd3fe8beded8d35679ece9928fdd649048a47bd0aa4b982eb02f3d8db919220 not found: ID does not exist" Jul 10 19:16:51.430180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:51.430140 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8df697c9f-lmhgn"] Jul 10 19:16:51.435798 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:51.435766 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8df697c9f-lmhgn"] Jul 10 19:16:52.517093 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:52.517059 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb053c11-b5a2-440a-a52b-4ddcc3dc8459" path="/var/lib/kubelet/pods/cb053c11-b5a2-440a-a52b-4ddcc3dc8459/volumes" Jul 10 19:16:54.741454 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.741415 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65bd7478d4-rtpxr"] Jul 10 19:16:54.741846 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.741817 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb053c11-b5a2-440a-a52b-4ddcc3dc8459" containerName="authorino" Jul 10 19:16:54.741846 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.741828 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb053c11-b5a2-440a-a52b-4ddcc3dc8459" containerName="authorino" Jul 10 19:16:54.741975 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.741921 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb053c11-b5a2-440a-a52b-4ddcc3dc8459" containerName="authorino" Jul 10 19:16:54.746397 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.746370 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:16:54.749616 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.749589 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 19:16:54.750264 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.750237 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 19:16:54.750690 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.750672 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x87ft\"" Jul 10 19:16:54.756275 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.756242 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65bd7478d4-rtpxr"] Jul 10 19:16:54.904135 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.904098 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mqjl\" (UniqueName: \"kubernetes.io/projected/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-kube-api-access-8mqjl\") pod \"maas-controller-65bd7478d4-rtpxr\" (UID: \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\") " pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:16:54.904344 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.904172 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-webhook-cert\") pod \"maas-controller-65bd7478d4-rtpxr\" (UID: \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\") " pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:16:54.906858 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.906830 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6886c467c5-jscsf"] Jul 10 19:16:54.910659 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.910635 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:16:54.919662 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:54.919636 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6886c467c5-jscsf"] Jul 10 19:16:55.005329 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.005237 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg69x\" (UniqueName: \"kubernetes.io/projected/fb98a2e6-28e9-40be-9d34-7fd99da16528-kube-api-access-tg69x\") pod \"maas-controller-6886c467c5-jscsf\" (UID: \"fb98a2e6-28e9-40be-9d34-7fd99da16528\") " pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:16:55.005329 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.005289 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb98a2e6-28e9-40be-9d34-7fd99da16528-webhook-cert\") pod \"maas-controller-6886c467c5-jscsf\" (UID: \"fb98a2e6-28e9-40be-9d34-7fd99da16528\") " pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:16:55.005329 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.005326 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mqjl\" (UniqueName: \"kubernetes.io/projected/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-kube-api-access-8mqjl\") pod \"maas-controller-65bd7478d4-rtpxr\" (UID: \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\") " pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:16:55.005556 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.005386 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-webhook-cert\") pod \"maas-controller-65bd7478d4-rtpxr\" (UID: \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\") " pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:16:55.008029 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.007993 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-webhook-cert\") pod \"maas-controller-65bd7478d4-rtpxr\" (UID: \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\") " pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:16:55.013936 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.013910 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mqjl\" (UniqueName: \"kubernetes.io/projected/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-kube-api-access-8mqjl\") pod \"maas-controller-65bd7478d4-rtpxr\" (UID: \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\") " pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:16:55.058740 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.058699 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:16:55.106225 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.106188 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg69x\" (UniqueName: \"kubernetes.io/projected/fb98a2e6-28e9-40be-9d34-7fd99da16528-kube-api-access-tg69x\") pod \"maas-controller-6886c467c5-jscsf\" (UID: \"fb98a2e6-28e9-40be-9d34-7fd99da16528\") " pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:16:55.106225 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.106229 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb98a2e6-28e9-40be-9d34-7fd99da16528-webhook-cert\") pod \"maas-controller-6886c467c5-jscsf\" (UID: \"fb98a2e6-28e9-40be-9d34-7fd99da16528\") " pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:16:55.108927 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.108900 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb98a2e6-28e9-40be-9d34-7fd99da16528-webhook-cert\") pod \"maas-controller-6886c467c5-jscsf\" (UID: \"fb98a2e6-28e9-40be-9d34-7fd99da16528\") " pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:16:55.116363 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.116300 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg69x\" (UniqueName: \"kubernetes.io/projected/fb98a2e6-28e9-40be-9d34-7fd99da16528-kube-api-access-tg69x\") pod \"maas-controller-6886c467c5-jscsf\" (UID: \"fb98a2e6-28e9-40be-9d34-7fd99da16528\") " pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:16:55.208695 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.208662 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65bd7478d4-rtpxr"] Jul 10 19:16:55.210652 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:16:55.210625 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d4e8a2_04ae_4644_8eb3_41151a25c6b2.slice/crio-b962e7b5fb0c5d64d6ec9a7f4cced1febbcc7cb59352c658c3dfff050ae52c43 WatchSource:0}: Error finding container b962e7b5fb0c5d64d6ec9a7f4cced1febbcc7cb59352c658c3dfff050ae52c43: Status 404 returned error can't find the container with id b962e7b5fb0c5d64d6ec9a7f4cced1febbcc7cb59352c658c3dfff050ae52c43 Jul 10 19:16:55.222669 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.222637 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:16:55.370228 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.370191 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6886c467c5-jscsf"] Jul 10 19:16:55.372063 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:16:55.372027 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb98a2e6_28e9_40be_9d34_7fd99da16528.slice/crio-b1bd4e5f1818900de722f761f1cfd42ee7c9bbbc384e06a332465fc7eee5b5e8 WatchSource:0}: Error finding container b1bd4e5f1818900de722f761f1cfd42ee7c9bbbc384e06a332465fc7eee5b5e8: Status 404 returned error can't find the container with id b1bd4e5f1818900de722f761f1cfd42ee7c9bbbc384e06a332465fc7eee5b5e8 Jul 10 19:16:55.427525 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.427485 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886c467c5-jscsf" event={"ID":"fb98a2e6-28e9-40be-9d34-7fd99da16528","Type":"ContainerStarted","Data":"b1bd4e5f1818900de722f761f1cfd42ee7c9bbbc384e06a332465fc7eee5b5e8"} Jul 10 19:16:55.428669 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:55.428644 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" event={"ID":"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2","Type":"ContainerStarted","Data":"b962e7b5fb0c5d64d6ec9a7f4cced1febbcc7cb59352c658c3dfff050ae52c43"} Jul 10 19:16:58.453651 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:58.453586 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886c467c5-jscsf" event={"ID":"fb98a2e6-28e9-40be-9d34-7fd99da16528","Type":"ContainerStarted","Data":"e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b"} Jul 10 19:16:58.454205 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:58.453677 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:16:58.455324 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:58.455291 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" event={"ID":"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2","Type":"ContainerStarted","Data":"c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2"} Jul 10 19:16:58.455465 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:58.455414 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:16:58.471716 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:58.471658 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6886c467c5-jscsf" podStartSLOduration=2.399762239 podStartE2EDuration="4.471642672s" totalImagesPullingTime="2.071880433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:16:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:16:58.469942426 +0000 UTC m=+570.486400798" watchObservedRunningTime="2026-07-10 19:16:58.471642672 +0000 UTC m=+570.488101043" Jul 10 19:16:58.488926 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:16:58.488843 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" podStartSLOduration=2.2661965 podStartE2EDuration="4.48882523s" totalImagesPullingTime="2.22262873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:16:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:16:58.488279722 +0000 UTC m=+570.504738109" watchObservedRunningTime="2026-07-10 19:16:58.48882523 +0000 UTC m=+570.505283603" Jul 10 19:17:09.467172 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.467117 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:17:09.467664 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.467641 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:17:09.528105 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.528068 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65bd7478d4-rtpxr"] Jul 10 19:17:09.528550 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.528516 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" podUID="b1d4e8a2-04ae-4644-8eb3-41151a25c6b2" containerName="manager" containerID="cri-o://c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2" gracePeriod=30 Jul 10 19:17:09.791584 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.791555 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:17:09.839568 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.839534 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-webhook-cert\") pod \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\" (UID: \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\") " Jul 10 19:17:09.839761 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.839593 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mqjl\" (UniqueName: \"kubernetes.io/projected/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-kube-api-access-8mqjl\") pod \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\" (UID: \"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2\") " Jul 10 19:17:09.842018 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.841983 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b1d4e8a2-04ae-4644-8eb3-41151a25c6b2" (UID: "b1d4e8a2-04ae-4644-8eb3-41151a25c6b2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:17:09.842154 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.842039 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-kube-api-access-8mqjl" (OuterVolumeSpecName: "kube-api-access-8mqjl") pod "b1d4e8a2-04ae-4644-8eb3-41151a25c6b2" (UID: "b1d4e8a2-04ae-4644-8eb3-41151a25c6b2"). InnerVolumeSpecName "kube-api-access-8mqjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:17:09.940681 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.940646 2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-webhook-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:17:09.940681 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:09.940678 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mqjl\" (UniqueName: \"kubernetes.io/projected/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2-kube-api-access-8mqjl\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:17:10.524505 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:10.524473 2556 generic.go:356] "Generic (PLEG): container finished" podID="b1d4e8a2-04ae-4644-8eb3-41151a25c6b2" containerID="c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2" exitCode=0 Jul 10 19:17:10.524966 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:10.524533 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" Jul 10 19:17:10.524966 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:10.524585 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" event={"ID":"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2","Type":"ContainerDied","Data":"c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2"} Jul 10 19:17:10.524966 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:10.524634 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65bd7478d4-rtpxr" event={"ID":"b1d4e8a2-04ae-4644-8eb3-41151a25c6b2","Type":"ContainerDied","Data":"b962e7b5fb0c5d64d6ec9a7f4cced1febbcc7cb59352c658c3dfff050ae52c43"} Jul 10 19:17:10.524966 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:10.524662 2556 scope.go:117] "RemoveContainer" containerID="c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2" Jul 10 19:17:10.534641 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:10.534457 2556 scope.go:117] "RemoveContainer" containerID="c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2" Jul 10 19:17:10.534810 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:17:10.534789 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2\": container with ID starting with c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2 not found: ID does not exist" containerID="c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2" Jul 10 19:17:10.534885 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:10.534822 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2"} err="failed to get container status \"c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2\": rpc error: code = NotFound desc = could not find container \"c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2\": container with ID starting with c7238c1595908a24fc1a51330d74e96fd6a840746a23eae57343fa562f1a9fe2 not found: ID does not exist" Jul 10 19:17:10.541719 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:10.541690 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65bd7478d4-rtpxr"] Jul 10 19:17:10.544125 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:10.544097 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65bd7478d4-rtpxr"] Jul 10 19:17:12.517915 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:12.517833 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1d4e8a2-04ae-4644-8eb3-41151a25c6b2" path="/var/lib/kubelet/pods/b1d4e8a2-04ae-4644-8eb3-41151a25c6b2/volumes" Jul 10 19:17:28.256799 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.256758 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mnfxk"] Jul 10 19:17:28.257420 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.257400 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d4e8a2-04ae-4644-8eb3-41151a25c6b2" containerName="manager" Jul 10 19:17:28.257484 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.257423 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d4e8a2-04ae-4644-8eb3-41151a25c6b2" containerName="manager" Jul 10 19:17:28.257534 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.257515 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d4e8a2-04ae-4644-8eb3-41151a25c6b2" containerName="manager" Jul 10 19:17:28.263102 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.263079 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:28.267223 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.267191 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6wkvj\"" Jul 10 19:17:28.268104 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.268074 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 19:17:28.272916 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.271107 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mnfxk"] Jul 10 19:17:28.314539 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.314500 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6a7004c-8886-4a50-a427-c64ef9aec704-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mnfxk\" (UID: \"b6a7004c-8886-4a50-a427-c64ef9aec704\") " pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:28.314717 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.314580 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9hwx\" (UniqueName: \"kubernetes.io/projected/b6a7004c-8886-4a50-a427-c64ef9aec704-kube-api-access-k9hwx\") pod \"payload-pre-processing-7964457696-mnfxk\" (UID: \"b6a7004c-8886-4a50-a427-c64ef9aec704\") " pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:28.415107 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.415079 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9hwx\" (UniqueName: \"kubernetes.io/projected/b6a7004c-8886-4a50-a427-c64ef9aec704-kube-api-access-k9hwx\") pod \"payload-pre-processing-7964457696-mnfxk\" (UID: \"b6a7004c-8886-4a50-a427-c64ef9aec704\") " pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:28.415307 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.415200 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6a7004c-8886-4a50-a427-c64ef9aec704-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mnfxk\" (UID: \"b6a7004c-8886-4a50-a427-c64ef9aec704\") " pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:28.417703 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.417665 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 19:17:28.419863 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.419832 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:17:28.423796 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.423768 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:17:28.424007 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.423911 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9hwx\" (UniqueName: \"kubernetes.io/projected/b6a7004c-8886-4a50-a427-c64ef9aec704-kube-api-access-k9hwx\") pod \"payload-pre-processing-7964457696-mnfxk\" (UID: \"b6a7004c-8886-4a50-a427-c64ef9aec704\") " pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:28.424733 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.424712 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:17:28.426099 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.426072 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6a7004c-8886-4a50-a427-c64ef9aec704-plugins-config-volume\") pod \"payload-pre-processing-7964457696-mnfxk\" (UID: \"b6a7004c-8886-4a50-a427-c64ef9aec704\") " pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:28.428974 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.428949 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:17:28.585948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.585815 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6wkvj\"" Jul 10 19:17:28.593942 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.593860 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:28.765725 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:28.765683 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-mnfxk"] Jul 10 19:17:28.784128 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:17:28.784091 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a7004c_8886_4a50_a427_c64ef9aec704.slice/crio-1e302fdd596b831e103dad60b0d1267ac00b7295a1f13329b35e8003579e6b8f WatchSource:0}: Error finding container 1e302fdd596b831e103dad60b0d1267ac00b7295a1f13329b35e8003579e6b8f: Status 404 returned error can't find the container with id 1e302fdd596b831e103dad60b0d1267ac00b7295a1f13329b35e8003579e6b8f Jul 10 19:17:29.644613 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:29.644573 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" event={"ID":"b6a7004c-8886-4a50-a427-c64ef9aec704","Type":"ContainerStarted","Data":"1e302fdd596b831e103dad60b0d1267ac00b7295a1f13329b35e8003579e6b8f"} Jul 10 19:17:30.652555 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:30.652451 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" event={"ID":"b6a7004c-8886-4a50-a427-c64ef9aec704","Type":"ContainerStarted","Data":"371d2cd50b5091c50e0fd6977280afad865e45fe53c455e480598e1d1e6c5dce"} Jul 10 19:17:30.653001 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:30.652584 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:30.669942 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:30.669849 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" podStartSLOduration=1.107882282 podStartE2EDuration="2.669832342s" totalImagesPullingTime="1.56195006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:17:30.668229829 +0000 UTC m=+602.684688200" watchObservedRunningTime="2026-07-10 19:17:30.669832342 +0000 UTC m=+602.686290713" Jul 10 19:17:37.697014 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:37.696978 2556 generic.go:356] "Generic (PLEG): container finished" podID="821b101f-507a-422a-83c3-58207b76e691" containerID="e6048b826098e7af6827794fc5e24af371aded2cd13cec3ce525d9de97797c27" exitCode=0 Jul 10 19:17:37.697428 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:37.697053 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" event={"ID":"821b101f-507a-422a-83c3-58207b76e691","Type":"ContainerDied","Data":"e6048b826098e7af6827794fc5e24af371aded2cd13cec3ce525d9de97797c27"} Jul 10 19:17:37.697472 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:37.697437 2556 scope.go:117] "RemoveContainer" containerID="e6048b826098e7af6827794fc5e24af371aded2cd13cec3ce525d9de97797c27" Jul 10 19:17:37.698578 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:37.698551 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:17:37.698646 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:37.698625 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:17:37.698683 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:37.698665 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 10 19:17:38.704274 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:38.704236 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" event={"ID":"821b101f-507a-422a-83c3-58207b76e691","Type":"ContainerStarted","Data":"ae2b04e561071ea5c5de25e0bb94d3398648f3f30486669426c7b12cfe95fc2e"} Jul 10 19:17:38.956092 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:38.955995 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:17:38.956294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:38.956266 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sc64n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 10 19:17:38.956381 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:38.956326 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" podUID="821b101f-507a-422a-83c3-58207b76e691" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 10 19:17:39.955969 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:39.955928 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sc64n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 10 19:17:39.956374 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:39.955994 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" podUID="821b101f-507a-422a-83c3-58207b76e691" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 10 19:17:40.956235 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:40.956197 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sc64n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 10 19:17:40.956617 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:40.956253 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" podUID="821b101f-507a-422a-83c3-58207b76e691" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 10 19:17:41.660840 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:41.660804 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-mnfxk" Jul 10 19:17:41.956251 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:41.956144 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sc64n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 10 19:17:41.956251 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:41.956203 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" podUID="821b101f-507a-422a-83c3-58207b76e691" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 10 19:17:42.956598 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:42.956559 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sc64n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 10 19:17:42.957032 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:42.956617 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" podUID="821b101f-507a-422a-83c3-58207b76e691" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 10 19:17:43.956817 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:43.956773 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sc64n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 10 19:17:43.957216 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:43.956833 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" podUID="821b101f-507a-422a-83c3-58207b76e691" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 10 19:17:44.179759 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:44.179719 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:17:44.960026 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:44.959994 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:17:44.961148 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:17:44.961121 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sc64n" Jul 10 19:18:06.383238 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.383201 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7"] Jul 10 19:18:06.387574 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.387552 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.390014 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.389988 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 19:18:06.391053 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.391037 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mmzcw\"" Jul 10 19:18:06.391143 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.391078 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 10 19:18:06.391143 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.391095 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 19:18:06.395732 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.395702 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7"] Jul 10 19:18:06.466985 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.466947 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287918a2-0013-4a2f-9431-f0d0dba40b92-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.467189 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.466996 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.467189 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.467084 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zjns\" (UniqueName: \"kubernetes.io/projected/287918a2-0013-4a2f-9431-f0d0dba40b92-kube-api-access-4zjns\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.467189 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.467125 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.467189 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.467160 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.467377 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.467195 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.567846 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.567801 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287918a2-0013-4a2f-9431-f0d0dba40b92-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.567846 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.567847 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.568119 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.567907 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zjns\" (UniqueName: \"kubernetes.io/projected/287918a2-0013-4a2f-9431-f0d0dba40b92-kube-api-access-4zjns\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.568119 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.567931 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.568119 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.567954 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.568119 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.567979 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.568419 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.568396 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.568526 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.568430 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.568526 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.568467 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.570496 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.570478 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/287918a2-0013-4a2f-9431-f0d0dba40b92-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.570794 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.570774 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287918a2-0013-4a2f-9431-f0d0dba40b92-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.576997 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.576966 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zjns\" (UniqueName: \"kubernetes.io/projected/287918a2-0013-4a2f-9431-f0d0dba40b92-kube-api-access-4zjns\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7\" (UID: \"287918a2-0013-4a2f-9431-f0d0dba40b92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.700147 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.700048 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:06.864207 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.863006 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7"] Jul 10 19:18:06.868932 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.868137 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:18:06.878702 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:06.878667 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" event={"ID":"287918a2-0013-4a2f-9431-f0d0dba40b92","Type":"ContainerStarted","Data":"627032e99ef005357fd5b21308c74bb07ff301c4d5c6ae734b79c625682c577a"} Jul 10 19:18:08.893479 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:08.893442 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" event={"ID":"287918a2-0013-4a2f-9431-f0d0dba40b92","Type":"ContainerStarted","Data":"db79145f49962322c9d20775bcd39368927782bd614bd8569f44646d66f1dc16"} Jul 10 19:18:08.909746 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:08.909682 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" podStartSLOduration=1.613280687 podStartE2EDuration="2.909664623s" totalImagesPullingTime="1.296383936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:18:08.908706903 +0000 UTC m=+640.925165298" watchObservedRunningTime="2026-07-10 19:18:08.909664623 +0000 UTC m=+640.926122994" Jul 10 19:18:16.700543 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:16.700498 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:16.700978 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:16.700574 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:16.713495 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:16.713466 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:16.957188 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:16.957100 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7" Jul 10 19:18:55.554681 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.554644 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh"] Jul 10 19:18:55.558365 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.558335 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.560851 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.560827 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 10 19:18:55.566818 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.566780 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh"] Jul 10 19:18:55.733786 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.733740 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.733786 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.733791 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd6a4418-4caa-4105-bf9b-f37037618989-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.734054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.733815 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq6p5\" (UniqueName: \"kubernetes.io/projected/fd6a4418-4caa-4105-bf9b-f37037618989-kube-api-access-vq6p5\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.734054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.733911 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.734054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.733952 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.734054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.733985 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.835288 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.835170 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.835288 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.835233 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd6a4418-4caa-4105-bf9b-f37037618989-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.835288 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.835256 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq6p5\" (UniqueName: \"kubernetes.io/projected/fd6a4418-4caa-4105-bf9b-f37037618989-kube-api-access-vq6p5\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.835288 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.835286 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.835288 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.835303 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.835635 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.835325 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.836180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.836142 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.836334 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.836278 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.836399 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.836364 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.843912 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.839278 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd6a4418-4caa-4105-bf9b-f37037618989-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.843912 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.839544 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd6a4418-4caa-4105-bf9b-f37037618989-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.845055 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.845014 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq6p5\" (UniqueName: \"kubernetes.io/projected/fd6a4418-4caa-4105-bf9b-f37037618989-kube-api-access-vq6p5\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2dhhh\" (UID: \"fd6a4418-4caa-4105-bf9b-f37037618989\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:55.873804 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:55.873754 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:18:56.029628 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:56.029587 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh"] Jul 10 19:18:56.030619 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:18:56.030578 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd6a4418_4caa_4105_bf9b_f37037618989.slice/crio-e19cf917816feec8a38e4a3839e5621b5305d567c67dd80effa0106ac557ddce WatchSource:0}: Error finding container e19cf917816feec8a38e4a3839e5621b5305d567c67dd80effa0106ac557ddce: Status 404 returned error can't find the container with id e19cf917816feec8a38e4a3839e5621b5305d567c67dd80effa0106ac557ddce Jul 10 19:18:56.170554 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:56.170465 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" event={"ID":"fd6a4418-4caa-4105-bf9b-f37037618989","Type":"ContainerStarted","Data":"e19cf917816feec8a38e4a3839e5621b5305d567c67dd80effa0106ac557ddce"} Jul 10 19:18:57.177918 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:57.177864 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" event={"ID":"fd6a4418-4caa-4105-bf9b-f37037618989","Type":"ContainerStarted","Data":"bb6824eed52ccbaadb3208b992d404371176d87cba839693791a2ff7b46ccdc6"} Jul 10 19:18:57.197018 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:18:57.196960 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" podStartSLOduration=1.994498984 podStartE2EDuration="2.19694307s" totalImagesPullingTime="202.444086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:18:57.194084765 +0000 UTC m=+689.210543159" watchObservedRunningTime="2026-07-10 19:18:57.19694307 +0000 UTC m=+689.213401440" Jul 10 19:19:05.874507 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:05.874469 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:19:05.874940 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:05.874563 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:19:05.887436 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:05.887406 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:19:06.243423 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:06.243396 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2dhhh" Jul 10 19:19:39.762830 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.762729 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4"] Jul 10 19:19:39.766836 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.766813 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:39.769142 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.769119 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 10 19:19:39.776131 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.776104 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4"] Jul 10 19:19:39.945445 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.945406 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:39.945639 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.945470 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:39.945639 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.945529 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99rmj\" (UniqueName: \"kubernetes.io/projected/4e19fc92-05d3-453c-9165-557b9e4fc75d-kube-api-access-99rmj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:39.945639 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.945590 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e19fc92-05d3-453c-9165-557b9e4fc75d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:39.945788 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.945682 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:39.945788 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:39.945712 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.046630 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.046532 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.046630 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.046582 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99rmj\" (UniqueName: \"kubernetes.io/projected/4e19fc92-05d3-453c-9165-557b9e4fc75d-kube-api-access-99rmj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.046630 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.046623 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e19fc92-05d3-453c-9165-557b9e4fc75d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.046949 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.046684 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.046949 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.046845 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.046949 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.046942 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.047114 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.047050 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.047114 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.047086 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.047208 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.047138 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.049234 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.049212 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e19fc92-05d3-453c-9165-557b9e4fc75d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.049499 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.049479 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e19fc92-05d3-453c-9165-557b9e4fc75d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.054827 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.054797 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99rmj\" (UniqueName: \"kubernetes.io/projected/4e19fc92-05d3-453c-9165-557b9e4fc75d-kube-api-access-99rmj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4\" (UID: \"4e19fc92-05d3-453c-9165-557b9e4fc75d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.077608 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.077568 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:40.226160 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.226128 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4"] Jul 10 19:19:40.226864 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:19:40.226839 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e19fc92_05d3_453c_9165_557b9e4fc75d.slice/crio-9ef7eee60e37842298d21603b9ca35f154ef812b2dd9461ac2a9b0e1bdb92ec5 WatchSource:0}: Error finding container 9ef7eee60e37842298d21603b9ca35f154ef812b2dd9461ac2a9b0e1bdb92ec5: Status 404 returned error can't find the container with id 9ef7eee60e37842298d21603b9ca35f154ef812b2dd9461ac2a9b0e1bdb92ec5 Jul 10 19:19:40.438090 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:40.438050 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" event={"ID":"4e19fc92-05d3-453c-9165-557b9e4fc75d","Type":"ContainerStarted","Data":"9ef7eee60e37842298d21603b9ca35f154ef812b2dd9461ac2a9b0e1bdb92ec5"} Jul 10 19:19:41.446069 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:41.446028 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" event={"ID":"4e19fc92-05d3-453c-9165-557b9e4fc75d","Type":"ContainerStarted","Data":"85bb952b1d6c59391d341d608f7aba26d8931638a79c3e297b0bb22400da2508"} Jul 10 19:19:41.464685 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:41.464626 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" podStartSLOduration=2.286829961 podStartE2EDuration="2.464608362s" totalImagesPullingTime="177.778401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:19:41.462051656 +0000 UTC m=+733.478510033" watchObservedRunningTime="2026-07-10 19:19:41.464608362 +0000 UTC m=+733.481066733" Jul 10 19:19:50.078621 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:50.078573 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:50.079169 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:50.078646 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:50.091459 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:50.091432 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:19:50.509267 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:19:50.509235 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4" Jul 10 19:20:30.925723 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:30.925668 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55b5d99cb5-lbhbc"] Jul 10 19:20:30.929665 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:30.929639 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" Jul 10 19:20:30.940370 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:30.940329 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55b5d99cb5-lbhbc"] Jul 10 19:20:31.035239 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:31.035205 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7dc4ad0b-63ec-47e6-bfb0-eace2ed50998-tls-cert\") pod \"authorino-55b5d99cb5-lbhbc\" (UID: \"7dc4ad0b-63ec-47e6-bfb0-eace2ed50998\") " pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" Jul 10 19:20:31.035429 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:31.035255 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks5vr\" (UniqueName: \"kubernetes.io/projected/7dc4ad0b-63ec-47e6-bfb0-eace2ed50998-kube-api-access-ks5vr\") pod \"authorino-55b5d99cb5-lbhbc\" (UID: \"7dc4ad0b-63ec-47e6-bfb0-eace2ed50998\") " pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" Jul 10 19:20:31.136056 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:31.136019 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7dc4ad0b-63ec-47e6-bfb0-eace2ed50998-tls-cert\") pod \"authorino-55b5d99cb5-lbhbc\" (UID: \"7dc4ad0b-63ec-47e6-bfb0-eace2ed50998\") " pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" Jul 10 19:20:31.136245 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:31.136074 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks5vr\" (UniqueName: \"kubernetes.io/projected/7dc4ad0b-63ec-47e6-bfb0-eace2ed50998-kube-api-access-ks5vr\") pod \"authorino-55b5d99cb5-lbhbc\" (UID: \"7dc4ad0b-63ec-47e6-bfb0-eace2ed50998\") " pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" Jul 10 19:20:31.138861 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:31.138822 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7dc4ad0b-63ec-47e6-bfb0-eace2ed50998-tls-cert\") pod \"authorino-55b5d99cb5-lbhbc\" (UID: \"7dc4ad0b-63ec-47e6-bfb0-eace2ed50998\") " pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" Jul 10 19:20:31.143401 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:31.143373 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks5vr\" (UniqueName: \"kubernetes.io/projected/7dc4ad0b-63ec-47e6-bfb0-eace2ed50998-kube-api-access-ks5vr\") pod \"authorino-55b5d99cb5-lbhbc\" (UID: \"7dc4ad0b-63ec-47e6-bfb0-eace2ed50998\") " pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" Jul 10 19:20:31.248554 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:31.248513 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" Jul 10 19:20:31.394684 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:31.394654 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55b5d99cb5-lbhbc"] Jul 10 19:20:31.396523 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:20:31.396493 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7dc4ad0b_63ec_47e6_bfb0_eace2ed50998.slice/crio-47f9621a07d8d1428875803562bd4a296b614bc817f067c4e32be33d42cf9561 WatchSource:0}: Error finding container 47f9621a07d8d1428875803562bd4a296b614bc817f067c4e32be33d42cf9561: Status 404 returned error can't find the container with id 47f9621a07d8d1428875803562bd4a296b614bc817f067c4e32be33d42cf9561 Jul 10 19:20:31.742264 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:31.742226 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" event={"ID":"7dc4ad0b-63ec-47e6-bfb0-eace2ed50998","Type":"ContainerStarted","Data":"47f9621a07d8d1428875803562bd4a296b614bc817f067c4e32be33d42cf9561"} Jul 10 19:20:32.752823 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:32.752781 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" event={"ID":"7dc4ad0b-63ec-47e6-bfb0-eace2ed50998","Type":"ContainerStarted","Data":"e182bdcb40cb8aae6f1b7d85421184b7021bfd5a94ba252d0a95bc6cc8ea6a2c"} Jul 10 19:20:32.772128 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:32.772072 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55b5d99cb5-lbhbc" podStartSLOduration=2.035451127 podStartE2EDuration="2.77205106s" totalImagesPullingTime="736.599933ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:20:32.77097259 +0000 UTC m=+784.787430964" watchObservedRunningTime="2026-07-10 19:20:32.77205106 +0000 UTC m=+784.788509431" Jul 10 19:20:32.799079 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:32.799044 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-655655858d-pqpz7"] Jul 10 19:20:32.799293 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:32.799245 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-655655858d-pqpz7" podUID="0f0e2c75-cf1a-46df-b681-ffabbd9261cf" containerName="authorino" containerID="cri-o://0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa" gracePeriod=30 Jul 10 19:20:33.081286 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.081254 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:20:33.153215 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.153178 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwgd9\" (UniqueName: \"kubernetes.io/projected/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-kube-api-access-xwgd9\") pod \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\" (UID: \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\") " Jul 10 19:20:33.153408 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.153263 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-tls-cert\") pod \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\" (UID: \"0f0e2c75-cf1a-46df-b681-ffabbd9261cf\") " Jul 10 19:20:33.155502 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.155464 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-kube-api-access-xwgd9" (OuterVolumeSpecName: "kube-api-access-xwgd9") pod "0f0e2c75-cf1a-46df-b681-ffabbd9261cf" (UID: "0f0e2c75-cf1a-46df-b681-ffabbd9261cf"). InnerVolumeSpecName "kube-api-access-xwgd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:20:33.168147 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.168110 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0f0e2c75-cf1a-46df-b681-ffabbd9261cf" (UID: "0f0e2c75-cf1a-46df-b681-ffabbd9261cf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:20:33.254798 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.254756 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwgd9\" (UniqueName: \"kubernetes.io/projected/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-kube-api-access-xwgd9\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:20:33.254798 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.254801 2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f0e2c75-cf1a-46df-b681-ffabbd9261cf-tls-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:20:33.760924 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.760846 2556 generic.go:356] "Generic (PLEG): container finished" podID="0f0e2c75-cf1a-46df-b681-ffabbd9261cf" containerID="0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa" exitCode=0 Jul 10 19:20:33.761430 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.760929 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-655655858d-pqpz7" Jul 10 19:20:33.761430 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.760941 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-655655858d-pqpz7" event={"ID":"0f0e2c75-cf1a-46df-b681-ffabbd9261cf","Type":"ContainerDied","Data":"0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa"} Jul 10 19:20:33.761430 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.761008 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-655655858d-pqpz7" event={"ID":"0f0e2c75-cf1a-46df-b681-ffabbd9261cf","Type":"ContainerDied","Data":"f415167a8efac971fc7465cee67079c7b72229cbd4f221b6b52a5a7c4619c4dc"} Jul 10 19:20:33.761430 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.761033 2556 scope.go:117] "RemoveContainer" containerID="0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa" Jul 10 19:20:33.772606 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.772583 2556 scope.go:117] "RemoveContainer" containerID="0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa" Jul 10 19:20:33.772951 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:20:33.772925 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa\": container with ID starting with 0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa not found: ID does not exist" containerID="0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa" Jul 10 19:20:33.773037 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.772959 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa"} err="failed to get container status \"0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa\": rpc error: code = NotFound desc = could not find container \"0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa\": container with ID starting with 0d5d6f4f21de069ff9c462d3f9cba89fe42c6b08b9da3b90ec04eb75781ecafa not found: ID does not exist" Jul 10 19:20:33.788957 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.788918 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-655655858d-pqpz7"] Jul 10 19:20:33.792912 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:33.792862 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-655655858d-pqpz7"] Jul 10 19:20:34.519021 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:20:34.518991 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f0e2c75-cf1a-46df-b681-ffabbd9261cf" path="/var/lib/kubelet/pods/0f0e2c75-cf1a-46df-b681-ffabbd9261cf/volumes" Jul 10 19:21:44.337509 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:44.337472 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6886c467c5-jscsf"] Jul 10 19:21:44.338077 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:44.337701 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6886c467c5-jscsf" podUID="fb98a2e6-28e9-40be-9d34-7fd99da16528" containerName="manager" containerID="cri-o://e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b" gracePeriod=30 Jul 10 19:21:44.602578 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:44.602495 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:21:44.743825 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:44.743788 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb98a2e6-28e9-40be-9d34-7fd99da16528-webhook-cert\") pod \"fb98a2e6-28e9-40be-9d34-7fd99da16528\" (UID: \"fb98a2e6-28e9-40be-9d34-7fd99da16528\") " Jul 10 19:21:44.744063 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:44.743939 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg69x\" (UniqueName: \"kubernetes.io/projected/fb98a2e6-28e9-40be-9d34-7fd99da16528-kube-api-access-tg69x\") pod \"fb98a2e6-28e9-40be-9d34-7fd99da16528\" (UID: \"fb98a2e6-28e9-40be-9d34-7fd99da16528\") " Jul 10 19:21:44.746135 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:44.746100 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb98a2e6-28e9-40be-9d34-7fd99da16528-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fb98a2e6-28e9-40be-9d34-7fd99da16528" (UID: "fb98a2e6-28e9-40be-9d34-7fd99da16528"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 19:21:44.746256 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:44.746199 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb98a2e6-28e9-40be-9d34-7fd99da16528-kube-api-access-tg69x" (OuterVolumeSpecName: "kube-api-access-tg69x") pod "fb98a2e6-28e9-40be-9d34-7fd99da16528" (UID: "fb98a2e6-28e9-40be-9d34-7fd99da16528"). InnerVolumeSpecName "kube-api-access-tg69x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:21:44.844997 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:44.844942 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg69x\" (UniqueName: \"kubernetes.io/projected/fb98a2e6-28e9-40be-9d34-7fd99da16528-kube-api-access-tg69x\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:21:44.844997 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:44.844990 2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb98a2e6-28e9-40be-9d34-7fd99da16528-webhook-cert\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:21:45.199686 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.199654 2556 generic.go:356] "Generic (PLEG): container finished" podID="fb98a2e6-28e9-40be-9d34-7fd99da16528" containerID="e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b" exitCode=0 Jul 10 19:21:45.199894 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.199695 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886c467c5-jscsf" Jul 10 19:21:45.199894 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.199741 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886c467c5-jscsf" event={"ID":"fb98a2e6-28e9-40be-9d34-7fd99da16528","Type":"ContainerDied","Data":"e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b"} Jul 10 19:21:45.199894 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.199793 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886c467c5-jscsf" event={"ID":"fb98a2e6-28e9-40be-9d34-7fd99da16528","Type":"ContainerDied","Data":"b1bd4e5f1818900de722f761f1cfd42ee7c9bbbc384e06a332465fc7eee5b5e8"} Jul 10 19:21:45.199894 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.199820 2556 scope.go:117] "RemoveContainer" containerID="e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b" Jul 10 19:21:45.210275 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.210254 2556 scope.go:117] "RemoveContainer" containerID="e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b" Jul 10 19:21:45.210595 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:21:45.210577 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b\": container with ID starting with e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b not found: ID does not exist" containerID="e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b" Jul 10 19:21:45.210657 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.210609 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b"} err="failed to get container status \"e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b\": rpc error: code = NotFound desc = could not find container \"e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b\": container with ID starting with e41b73ade44bf6191e7899b8abf5d07c76e3fe78701e4442b7ba961eb418b30b not found: ID does not exist" Jul 10 19:21:45.221952 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.221914 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6886c467c5-jscsf"] Jul 10 19:21:45.227164 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.227133 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6886c467c5-jscsf"] Jul 10 19:21:45.964186 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.964138 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6886c467c5-6x9pm"] Jul 10 19:21:45.964864 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.964835 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0e2c75-cf1a-46df-b681-ffabbd9261cf" containerName="authorino" Jul 10 19:21:45.965020 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.964907 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0e2c75-cf1a-46df-b681-ffabbd9261cf" containerName="authorino" Jul 10 19:21:45.965020 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.964923 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb98a2e6-28e9-40be-9d34-7fd99da16528" containerName="manager" Jul 10 19:21:45.965020 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.964942 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb98a2e6-28e9-40be-9d34-7fd99da16528" containerName="manager" Jul 10 19:21:45.965198 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.965104 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0e2c75-cf1a-46df-b681-ffabbd9261cf" containerName="authorino" Jul 10 19:21:45.965198 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.965121 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb98a2e6-28e9-40be-9d34-7fd99da16528" containerName="manager" Jul 10 19:21:45.969761 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.969732 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:21:45.973157 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.973135 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 19:21:45.973294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.973137 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-x87ft\"" Jul 10 19:21:45.973294 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.973177 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 19:21:45.977820 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:45.977791 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6886c467c5-6x9pm"] Jul 10 19:21:46.058061 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:46.058024 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j78h9\" (UniqueName: \"kubernetes.io/projected/a8f0c329-6f7f-4427-b1ff-d127b84d1302-kube-api-access-j78h9\") pod \"maas-controller-6886c467c5-6x9pm\" (UID: \"a8f0c329-6f7f-4427-b1ff-d127b84d1302\") " pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:21:46.058239 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:46.058099 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8f0c329-6f7f-4427-b1ff-d127b84d1302-webhook-cert\") pod \"maas-controller-6886c467c5-6x9pm\" (UID: \"a8f0c329-6f7f-4427-b1ff-d127b84d1302\") " pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:21:46.159528 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:46.159480 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j78h9\" (UniqueName: \"kubernetes.io/projected/a8f0c329-6f7f-4427-b1ff-d127b84d1302-kube-api-access-j78h9\") pod \"maas-controller-6886c467c5-6x9pm\" (UID: \"a8f0c329-6f7f-4427-b1ff-d127b84d1302\") " pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:21:46.159733 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:46.159574 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8f0c329-6f7f-4427-b1ff-d127b84d1302-webhook-cert\") pod \"maas-controller-6886c467c5-6x9pm\" (UID: \"a8f0c329-6f7f-4427-b1ff-d127b84d1302\") " pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:21:46.162372 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:46.162349 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8f0c329-6f7f-4427-b1ff-d127b84d1302-webhook-cert\") pod \"maas-controller-6886c467c5-6x9pm\" (UID: \"a8f0c329-6f7f-4427-b1ff-d127b84d1302\") " pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:21:46.168013 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:46.167984 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j78h9\" (UniqueName: \"kubernetes.io/projected/a8f0c329-6f7f-4427-b1ff-d127b84d1302-kube-api-access-j78h9\") pod \"maas-controller-6886c467c5-6x9pm\" (UID: \"a8f0c329-6f7f-4427-b1ff-d127b84d1302\") " pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:21:46.283061 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:46.283014 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:21:46.431510 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:46.431482 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6886c467c5-6x9pm"] Jul 10 19:21:46.433006 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:21:46.432978 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f0c329_6f7f_4427_b1ff_d127b84d1302.slice/crio-28b62cfb9cedf7cb1c0fd46ae9aa56c7e41a93762861adc57d4b714f84e29207 WatchSource:0}: Error finding container 28b62cfb9cedf7cb1c0fd46ae9aa56c7e41a93762861adc57d4b714f84e29207: Status 404 returned error can't find the container with id 28b62cfb9cedf7cb1c0fd46ae9aa56c7e41a93762861adc57d4b714f84e29207 Jul 10 19:21:46.517699 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:46.517662 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb98a2e6-28e9-40be-9d34-7fd99da16528" path="/var/lib/kubelet/pods/fb98a2e6-28e9-40be-9d34-7fd99da16528/volumes" Jul 10 19:21:47.214753 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:47.214644 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886c467c5-6x9pm" event={"ID":"a8f0c329-6f7f-4427-b1ff-d127b84d1302","Type":"ContainerStarted","Data":"2e06d197abffc00878a2ce5ef2309e7201f0649cf634814207b30a3782d0b2f5"} Jul 10 19:21:47.214753 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:47.214705 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6886c467c5-6x9pm" event={"ID":"a8f0c329-6f7f-4427-b1ff-d127b84d1302","Type":"ContainerStarted","Data":"28b62cfb9cedf7cb1c0fd46ae9aa56c7e41a93762861adc57d4b714f84e29207"} Jul 10 19:21:47.214753 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:47.214726 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:21:47.237349 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:47.237291 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6886c467c5-6x9pm" podStartSLOduration=1.7742817739999999 podStartE2EDuration="2.237271415s" totalImagesPullingTime="462.989641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:21:47.235427727 +0000 UTC m=+859.251886219" watchObservedRunningTime="2026-07-10 19:21:47.237271415 +0000 UTC m=+859.253729827" Jul 10 19:21:58.226837 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:21:58.226802 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6886c467c5-6x9pm" Jul 10 19:22:28.467361 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:22:28.467279 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:22:28.471612 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:22:28.471581 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:22:28.475360 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:22:28.475287 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:22:28.480887 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:22:28.480847 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:26:28.462325 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:28.462278 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk"] Jul 10 19:26:28.462941 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:28.462514 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" podUID="12a765f6-b3ea-4fc7-ade3-9f10030c94f3" containerName="manager" containerID="cri-o://183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee" gracePeriod=10 Jul 10 19:26:29.832839 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.832807 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:26:29.887242 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.887146 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-765mt\" (UniqueName: \"kubernetes.io/projected/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-kube-api-access-765mt\") pod \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\" (UID: \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\") " Jul 10 19:26:29.887242 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.887206 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-extensions-socket-volume\") pod \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\" (UID: \"12a765f6-b3ea-4fc7-ade3-9f10030c94f3\") " Jul 10 19:26:29.887720 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.887687 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "12a765f6-b3ea-4fc7-ade3-9f10030c94f3" (UID: "12a765f6-b3ea-4fc7-ade3-9f10030c94f3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 19:26:29.889363 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.889336 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-kube-api-access-765mt" (OuterVolumeSpecName: "kube-api-access-765mt") pod "12a765f6-b3ea-4fc7-ade3-9f10030c94f3" (UID: "12a765f6-b3ea-4fc7-ade3-9f10030c94f3"). InnerVolumeSpecName "kube-api-access-765mt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:26:29.978287 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.978252 2556 generic.go:356] "Generic (PLEG): container finished" podID="12a765f6-b3ea-4fc7-ade3-9f10030c94f3" containerID="183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee" exitCode=0 Jul 10 19:26:29.978484 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.978301 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" Jul 10 19:26:29.978484 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.978321 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" event={"ID":"12a765f6-b3ea-4fc7-ade3-9f10030c94f3","Type":"ContainerDied","Data":"183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee"} Jul 10 19:26:29.978484 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.978366 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk" event={"ID":"12a765f6-b3ea-4fc7-ade3-9f10030c94f3","Type":"ContainerDied","Data":"3f0b7e98b65be01272115dac8722cf90ab218e1a5f768568ca500eb5b00a5c09"} Jul 10 19:26:29.978484 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.978389 2556 scope.go:117] "RemoveContainer" containerID="183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee" Jul 10 19:26:29.988052 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.988032 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-extensions-socket-volume\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:26:29.988052 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.988053 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-765mt\" (UniqueName: \"kubernetes.io/projected/12a765f6-b3ea-4fc7-ade3-9f10030c94f3-kube-api-access-765mt\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:26:29.990800 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.990784 2556 scope.go:117] "RemoveContainer" containerID="183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee" Jul 10 19:26:29.991229 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:26:29.991203 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee\": container with ID starting with 183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee not found: ID does not exist" containerID="183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee" Jul 10 19:26:29.991298 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:29.991238 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee"} err="failed to get container status \"183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee\": rpc error: code = NotFound desc = could not find container \"183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee\": container with ID starting with 183895bff9276cb262286d4e987dbf6e83601890cac71bb88724c1d8458e68ee not found: ID does not exist" Jul 10 19:26:30.002380 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:30.002342 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk"] Jul 10 19:26:30.006642 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:30.006600 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-bk8wk"] Jul 10 19:26:30.517585 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:26:30.517548 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12a765f6-b3ea-4fc7-ade3-9f10030c94f3" path="/var/lib/kubelet/pods/12a765f6-b3ea-4fc7-ade3-9f10030c94f3/volumes" Jul 10 19:27:28.513289 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:28.513256 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:27:28.517716 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:28.517688 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:27:28.520822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:28.520791 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:27:28.524283 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:28.524262 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:27:29.626274 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.626240 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf"] Jul 10 19:27:29.626683 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.626667 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12a765f6-b3ea-4fc7-ade3-9f10030c94f3" containerName="manager" Jul 10 19:27:29.626729 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.626684 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="12a765f6-b3ea-4fc7-ade3-9f10030c94f3" containerName="manager" Jul 10 19:27:29.626774 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.626765 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="12a765f6-b3ea-4fc7-ade3-9f10030c94f3" containerName="manager" Jul 10 19:27:29.630340 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.630322 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:27:29.633130 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.633108 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nzzvk\"" Jul 10 19:27:29.643285 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.643253 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf"] Jul 10 19:27:29.762918 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.762851 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c480bb25-68fa-4da6-b963-929728e4576a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7569bc7c69-sz6qf\" (UID: \"c480bb25-68fa-4da6-b963-929728e4576a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:27:29.763113 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.762951 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4s64\" (UniqueName: \"kubernetes.io/projected/c480bb25-68fa-4da6-b963-929728e4576a-kube-api-access-k4s64\") pod \"kuadrant-operator-controller-manager-7569bc7c69-sz6qf\" (UID: \"c480bb25-68fa-4da6-b963-929728e4576a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:27:29.864374 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.864329 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c480bb25-68fa-4da6-b963-929728e4576a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7569bc7c69-sz6qf\" (UID: \"c480bb25-68fa-4da6-b963-929728e4576a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:27:29.864586 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.864384 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4s64\" (UniqueName: \"kubernetes.io/projected/c480bb25-68fa-4da6-b963-929728e4576a-kube-api-access-k4s64\") pod \"kuadrant-operator-controller-manager-7569bc7c69-sz6qf\" (UID: \"c480bb25-68fa-4da6-b963-929728e4576a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:27:29.864790 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.864764 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c480bb25-68fa-4da6-b963-929728e4576a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7569bc7c69-sz6qf\" (UID: \"c480bb25-68fa-4da6-b963-929728e4576a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:27:29.877821 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.877745 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4s64\" (UniqueName: \"kubernetes.io/projected/c480bb25-68fa-4da6-b963-929728e4576a-kube-api-access-k4s64\") pod \"kuadrant-operator-controller-manager-7569bc7c69-sz6qf\" (UID: \"c480bb25-68fa-4da6-b963-929728e4576a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:27:29.942463 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:29.942434 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:27:30.097523 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:30.097494 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf"] Jul 10 19:27:30.098743 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:27:30.098712 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc480bb25_68fa_4da6_b963_929728e4576a.slice/crio-87102019a8b09bff5336b9a857d159ab217019f436460abd8393bab2d50a8d92 WatchSource:0}: Error finding container 87102019a8b09bff5336b9a857d159ab217019f436460abd8393bab2d50a8d92: Status 404 returned error can't find the container with id 87102019a8b09bff5336b9a857d159ab217019f436460abd8393bab2d50a8d92 Jul 10 19:27:30.324779 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:30.324746 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" event={"ID":"c480bb25-68fa-4da6-b963-929728e4576a","Type":"ContainerStarted","Data":"cffaa76d3576a4be9e0dbc968da39541235792e5737719a6447b4985bbb384a6"} Jul 10 19:27:30.324965 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:30.324804 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" event={"ID":"c480bb25-68fa-4da6-b963-929728e4576a","Type":"ContainerStarted","Data":"87102019a8b09bff5336b9a857d159ab217019f436460abd8393bab2d50a8d92"} Jul 10 19:27:30.324965 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:30.324825 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:27:30.346666 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:30.346616 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" podStartSLOduration=1.346600373 podStartE2EDuration="1.346600373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:27:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:27:30.344614461 +0000 UTC m=+1202.361072832" watchObservedRunningTime="2026-07-10 19:27:30.346600373 +0000 UTC m=+1202.363058744" Jul 10 19:27:41.333120 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:27:41.333089 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7569bc7c69-sz6qf" Jul 10 19:30:00.207578 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:00.207534 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk"] Jul 10 19:30:00.211104 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:00.211085 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" Jul 10 19:30:00.218489 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:00.218351 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8vsp\"" Jul 10 19:30:00.221560 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:00.221539 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" Jul 10 19:30:00.317113 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:00.317068 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk"] Jul 10 19:30:00.588074 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:00.588045 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk"] Jul 10 19:30:00.590166 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:30:00.590129 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183d8e80_ca54_4467_9f5b_d594798771b9.slice/crio-666b0d102aa799151a17cfbf495c843be6a9c0e4036b371614cea6dc0f37ac50 WatchSource:0}: Error finding container 666b0d102aa799151a17cfbf495c843be6a9c0e4036b371614cea6dc0f37ac50: Status 404 returned error can't find the container with id 666b0d102aa799151a17cfbf495c843be6a9c0e4036b371614cea6dc0f37ac50 Jul 10 19:30:00.592271 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:00.592254 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:30:01.233822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:01.233789 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" event={"ID":"183d8e80-ca54-4467-9f5b-d594798771b9","Type":"ContainerStarted","Data":"666b0d102aa799151a17cfbf495c843be6a9c0e4036b371614cea6dc0f37ac50"} Jul 10 19:30:04.255362 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:04.255318 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" event={"ID":"183d8e80-ca54-4467-9f5b-d594798771b9","Type":"ContainerStarted","Data":"f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b"} Jul 10 19:30:04.275305 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:30:04.275242 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" podStartSLOduration=1.8583721020000001 podStartE2EDuration="4.275220625s" totalImagesPullingTime="2.416848523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:30:04.273428611 +0000 UTC m=+1356.289886983" watchObservedRunningTime="2026-07-10 19:30:04.275220625 +0000 UTC m=+1356.291678996" Jul 10 19:31:00.011752 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:00.011707 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk"] Jul 10 19:31:00.012423 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:00.011970 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" podUID="183d8e80-ca54-4467-9f5b-d594798771b9" containerName="cleanup" containerID="cri-o://f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b" gracePeriod=30 Jul 10 19:31:30.207290 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.207258 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" Jul 10 19:31:30.783207 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.783113 2556 generic.go:356] "Generic (PLEG): container finished" podID="183d8e80-ca54-4467-9f5b-d594798771b9" containerID="f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b" exitCode=137 Jul 10 19:31:30.783207 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.783162 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" Jul 10 19:31:30.783411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.783199 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" event={"ID":"183d8e80-ca54-4467-9f5b-d594798771b9","Type":"ContainerDied","Data":"f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b"} Jul 10 19:31:30.783411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.783258 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk" event={"ID":"183d8e80-ca54-4467-9f5b-d594798771b9","Type":"ContainerDied","Data":"666b0d102aa799151a17cfbf495c843be6a9c0e4036b371614cea6dc0f37ac50"} Jul 10 19:31:30.783411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.783281 2556 scope.go:117] "RemoveContainer" containerID="f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b" Jul 10 19:31:30.793416 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.793391 2556 scope.go:117] "RemoveContainer" containerID="f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b" Jul 10 19:31:30.793760 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:31:30.793740 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b\": container with ID starting with f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b not found: ID does not exist" containerID="f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b" Jul 10 19:31:30.793817 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.793772 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b"} err="failed to get container status \"f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b\": rpc error: code = NotFound desc = could not find container \"f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b\": container with ID starting with f5afc5b7d3a8331d41e00596903397444c3675fc0a2c549789a925151c86885b not found: ID does not exist" Jul 10 19:31:30.813831 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.813795 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk"] Jul 10 19:31:30.816226 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:30.816197 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728530-9mljk"] Jul 10 19:31:32.518788 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:31:32.518753 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="183d8e80-ca54-4467-9f5b-d594798771b9" path="/var/lib/kubelet/pods/183d8e80-ca54-4467-9f5b-d594798771b9/volumes" Jul 10 19:32:28.555673 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:32:28.555639 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:32:28.559247 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:32:28.559220 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:32:28.566508 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:32:28.566480 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:32:28.569693 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:32:28.569666 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:37:28.605362 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:37:28.605316 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:37:28.605907 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:37:28.605712 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:37:28.608609 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:37:28.608589 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:37:28.608733 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:37:28.608659 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:42:28.645542 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:42:28.645510 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:42:28.646179 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:42:28.645923 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:42:28.648705 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:42:28.648682 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:42:28.649109 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:42:28.649092 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:45:00.154443 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.154403 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq"] Jul 10 19:45:00.155006 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.154893 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="183d8e80-ca54-4467-9f5b-d594798771b9" containerName="cleanup" Jul 10 19:45:00.155006 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.154913 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="183d8e80-ca54-4467-9f5b-d594798771b9" containerName="cleanup" Jul 10 19:45:00.155186 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.155067 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="183d8e80-ca54-4467-9f5b-d594798771b9" containerName="cleanup" Jul 10 19:45:00.158668 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.158642 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz"] Jul 10 19:45:00.158804 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.158779 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" Jul 10 19:45:00.161233 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.161209 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8vsp\"" Jul 10 19:45:00.162003 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.161985 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf"] Jul 10 19:45:00.162112 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.162093 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" Jul 10 19:45:00.165928 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.165902 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" Jul 10 19:45:00.167902 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.166929 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq"] Jul 10 19:45:00.171184 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.171158 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf"] Jul 10 19:45:00.173264 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.173241 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" Jul 10 19:45:00.181900 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.181855 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz"] Jul 10 19:45:00.184069 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.184043 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" Jul 10 19:45:00.222414 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.222386 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" Jul 10 19:45:00.358091 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.357994 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq"] Jul 10 19:45:00.360799 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:45:00.360761 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f73251c_7227_4222_89f7_bbd5b2154116.slice/crio-f9ad0be9b818aaa96ce25b37d7ae79221903cca56cc9305e2cfc934829adeb6a WatchSource:0}: Error finding container f9ad0be9b818aaa96ce25b37d7ae79221903cca56cc9305e2cfc934829adeb6a: Status 404 returned error can't find the container with id f9ad0be9b818aaa96ce25b37d7ae79221903cca56cc9305e2cfc934829adeb6a Jul 10 19:45:00.429447 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.429419 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf"] Jul 10 19:45:00.431032 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:45:00.431002 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61626e1e_b511_4d2e_9c48_72260766c07f.slice/crio-0326bce96f0985482dc2a7b9c0062e7bd8ef29f5486b628a4714dac779c9b2b2 WatchSource:0}: Error finding container 0326bce96f0985482dc2a7b9c0062e7bd8ef29f5486b628a4714dac779c9b2b2: Status 404 returned error can't find the container with id 0326bce96f0985482dc2a7b9c0062e7bd8ef29f5486b628a4714dac779c9b2b2 Jul 10 19:45:00.605901 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.605858 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz"] Jul 10 19:45:00.608828 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:45:00.608796 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd9e3c1e_4a66_4a17_a3e0_6342e9f1d9ee.slice/crio-2d2a4beaa454d071e967984356cb5e170998c2e6a1a656ac18b76d1f569f4f7a WatchSource:0}: Error finding container 2d2a4beaa454d071e967984356cb5e170998c2e6a1a656ac18b76d1f569f4f7a: Status 404 returned error can't find the container with id 2d2a4beaa454d071e967984356cb5e170998c2e6a1a656ac18b76d1f569f4f7a Jul 10 19:45:00.747347 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.747300 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" event={"ID":"61626e1e-b511-4d2e-9c48-72260766c07f","Type":"ContainerStarted","Data":"0326bce96f0985482dc2a7b9c0062e7bd8ef29f5486b628a4714dac779c9b2b2"} Jul 10 19:45:00.748569 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.748542 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" event={"ID":"bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee","Type":"ContainerStarted","Data":"2d2a4beaa454d071e967984356cb5e170998c2e6a1a656ac18b76d1f569f4f7a"} Jul 10 19:45:00.749643 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:00.749618 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" event={"ID":"8f73251c-7227-4222-89f7-bbd5b2154116","Type":"ContainerStarted","Data":"f9ad0be9b818aaa96ce25b37d7ae79221903cca56cc9305e2cfc934829adeb6a"} Jul 10 19:45:01.757377 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:01.757342 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" event={"ID":"61626e1e-b511-4d2e-9c48-72260766c07f","Type":"ContainerStarted","Data":"69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4"} Jul 10 19:45:01.758951 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:01.758918 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" event={"ID":"bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee","Type":"ContainerStarted","Data":"7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72"} Jul 10 19:45:01.760255 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:01.760231 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" event={"ID":"8f73251c-7227-4222-89f7-bbd5b2154116","Type":"ContainerStarted","Data":"04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992"} Jul 10 19:45:01.788686 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:01.788630 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" podStartSLOduration=1.78861337 podStartE2EDuration="1.78861337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:45:01.784291826 +0000 UTC m=+2253.800750197" watchObservedRunningTime="2026-07-10 19:45:01.78861337 +0000 UTC m=+2253.805071741" Jul 10 19:45:01.814357 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:01.814302 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" podStartSLOduration=1.814283848 podStartE2EDuration="1.814283848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:45:01.813195888 +0000 UTC m=+2253.829654263" watchObservedRunningTime="2026-07-10 19:45:01.814283848 +0000 UTC m=+2253.830742253" Jul 10 19:45:01.847360 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:45:01.847301 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" podStartSLOduration=1.847285559 podStartE2EDuration="1.847285559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:45:01.845816351 +0000 UTC m=+2253.862274723" watchObservedRunningTime="2026-07-10 19:45:01.847285559 +0000 UTC m=+2253.863743929" Jul 10 19:46:00.027543 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:00.027502 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf"] Jul 10 19:46:00.030056 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:00.027702 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" podUID="61626e1e-b511-4d2e-9c48-72260766c07f" containerName="cleanup" containerID="cri-o://69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4" gracePeriod=30 Jul 10 19:46:00.033233 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:00.033200 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq"] Jul 10 19:46:00.033485 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:00.033437 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" podUID="8f73251c-7227-4222-89f7-bbd5b2154116" containerName="cleanup" containerID="cri-o://04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992" gracePeriod=30 Jul 10 19:46:00.036114 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:00.036085 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz"] Jul 10 19:46:00.036287 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:00.036266 2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" podUID="bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee" containerName="cleanup" containerID="cri-o://7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72" gracePeriod=30 Jul 10 19:46:30.243505 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.243470 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" Jul 10 19:46:30.281177 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.281147 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" Jul 10 19:46:30.284566 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.284495 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" Jul 10 19:46:30.320951 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.320861 2556 generic.go:356] "Generic (PLEG): container finished" podID="61626e1e-b511-4d2e-9c48-72260766c07f" containerID="69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4" exitCode=137 Jul 10 19:46:30.321235 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.321045 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" event={"ID":"61626e1e-b511-4d2e-9c48-72260766c07f","Type":"ContainerDied","Data":"69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4"} Jul 10 19:46:30.321235 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.321097 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" event={"ID":"61626e1e-b511-4d2e-9c48-72260766c07f","Type":"ContainerDied","Data":"0326bce96f0985482dc2a7b9c0062e7bd8ef29f5486b628a4714dac779c9b2b2"} Jul 10 19:46:30.321235 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.321125 2556 scope.go:117] "RemoveContainer" containerID="69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4" Jul 10 19:46:30.322353 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.321465 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf" Jul 10 19:46:30.324553 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.324521 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee" containerID="7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72" exitCode=137 Jul 10 19:46:30.324700 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.324576 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" Jul 10 19:46:30.324700 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.324609 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" event={"ID":"bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee","Type":"ContainerDied","Data":"7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72"} Jul 10 19:46:30.324700 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.324671 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz" event={"ID":"bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee","Type":"ContainerDied","Data":"2d2a4beaa454d071e967984356cb5e170998c2e6a1a656ac18b76d1f569f4f7a"} Jul 10 19:46:30.327706 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.327670 2556 generic.go:356] "Generic (PLEG): container finished" podID="8f73251c-7227-4222-89f7-bbd5b2154116" containerID="04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992" exitCode=137 Jul 10 19:46:30.327864 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.327776 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" event={"ID":"8f73251c-7227-4222-89f7-bbd5b2154116","Type":"ContainerDied","Data":"04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992"} Jul 10 19:46:30.327864 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.327825 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" event={"ID":"8f73251c-7227-4222-89f7-bbd5b2154116","Type":"ContainerDied","Data":"f9ad0be9b818aaa96ce25b37d7ae79221903cca56cc9305e2cfc934829adeb6a"} Jul 10 19:46:30.328018 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.327918 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq" Jul 10 19:46:30.338016 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.337991 2556 scope.go:117] "RemoveContainer" containerID="69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4" Jul 10 19:46:30.338409 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:46:30.338384 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4\": container with ID starting with 69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4 not found: ID does not exist" containerID="69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4" Jul 10 19:46:30.338497 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.338421 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4"} err="failed to get container status \"69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4\": rpc error: code = NotFound desc = could not find container \"69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4\": container with ID starting with 69c3d345aeabecd39e60933d4bf3fdbe0b863fafa7be01f2b837bbb7865382d4 not found: ID does not exist" Jul 10 19:46:30.338497 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.338448 2556 scope.go:117] "RemoveContainer" containerID="7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72" Jul 10 19:46:30.349174 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.349144 2556 scope.go:117] "RemoveContainer" containerID="7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72" Jul 10 19:46:30.349693 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:46:30.349582 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72\": container with ID starting with 7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72 not found: ID does not exist" containerID="7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72" Jul 10 19:46:30.349693 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.349627 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72"} err="failed to get container status \"7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72\": rpc error: code = NotFound desc = could not find container \"7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72\": container with ID starting with 7ba8e383b7a698bd751767a258ce8835a772a44171fa42c84592bd6c1b720a72 not found: ID does not exist" Jul 10 19:46:30.349693 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.349656 2556 scope.go:117] "RemoveContainer" containerID="04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992" Jul 10 19:46:30.349991 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.349774 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf"] Jul 10 19:46:30.354548 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.354507 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-f25aa8-29728545-76jnf"] Jul 10 19:46:30.360327 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.360300 2556 scope.go:117] "RemoveContainer" containerID="04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992" Jul 10 19:46:30.361421 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:46:30.361395 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992\": container with ID starting with 04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992 not found: ID does not exist" containerID="04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992" Jul 10 19:46:30.361542 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.361428 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992"} err="failed to get container status \"04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992\": rpc error: code = NotFound desc = could not find container \"04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992\": container with ID starting with 04017b4bcaa950da6c349d5114911acc461deb5dbbc6bf77e2cba6aca9ae7992 not found: ID does not exist" Jul 10 19:46:30.365612 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.365585 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz"] Jul 10 19:46:30.368177 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.368155 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728545-l6zkz"] Jul 10 19:46:30.379762 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.379727 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq"] Jul 10 19:46:30.384144 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.384112 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-fc4b18-29728545-nhsmq"] Jul 10 19:46:30.519923 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.519885 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61626e1e-b511-4d2e-9c48-72260766c07f" path="/var/lib/kubelet/pods/61626e1e-b511-4d2e-9c48-72260766c07f/volumes" Jul 10 19:46:30.520215 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.520201 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f73251c-7227-4222-89f7-bbd5b2154116" path="/var/lib/kubelet/pods/8f73251c-7227-4222-89f7-bbd5b2154116/volumes" Jul 10 19:46:30.520450 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:46:30.520438 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee" path="/var/lib/kubelet/pods/bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee/volumes" Jul 10 19:47:11.560341 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560304 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7110-140324281430576"] Jul 10 19:47:11.560822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560726 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61626e1e-b511-4d2e-9c48-72260766c07f" containerName="cleanup" Jul 10 19:47:11.560822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560738 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="61626e1e-b511-4d2e-9c48-72260766c07f" containerName="cleanup" Jul 10 19:47:11.560822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560758 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee" containerName="cleanup" Jul 10 19:47:11.560822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560764 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee" containerName="cleanup" Jul 10 19:47:11.560822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560771 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f73251c-7227-4222-89f7-bbd5b2154116" containerName="cleanup" Jul 10 19:47:11.560822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560776 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f73251c-7227-4222-89f7-bbd5b2154116" containerName="cleanup" Jul 10 19:47:11.561054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560839 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="61626e1e-b511-4d2e-9c48-72260766c07f" containerName="cleanup" Jul 10 19:47:11.561054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560847 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd9e3c1e-4a66-4a17-a3e0-6342e9f1d9ee" containerName="cleanup" Jul 10 19:47:11.561054 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.560857 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f73251c-7227-4222-89f7-bbd5b2154116" containerName="cleanup" Jul 10 19:47:11.563632 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.563612 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324281430576" Jul 10 19:47:11.566516 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.566485 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-g9c6n\"" Jul 10 19:47:11.572829 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.572803 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324281430576"] Jul 10 19:47:11.612433 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.612388 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92znt\" (UniqueName: \"kubernetes.io/projected/5c30c984-108d-4412-8b0f-ecca9df7be99-kube-api-access-92znt\") pod \"test-curl-7110-140324281430576\" (UID: \"5c30c984-108d-4412-8b0f-ecca9df7be99\") " pod="openshift-ingress/test-curl-7110-140324281430576" Jul 10 19:47:11.713525 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.713488 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92znt\" (UniqueName: \"kubernetes.io/projected/5c30c984-108d-4412-8b0f-ecca9df7be99-kube-api-access-92znt\") pod \"test-curl-7110-140324281430576\" (UID: \"5c30c984-108d-4412-8b0f-ecca9df7be99\") " pod="openshift-ingress/test-curl-7110-140324281430576" Jul 10 19:47:11.725433 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.725396 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92znt\" (UniqueName: \"kubernetes.io/projected/5c30c984-108d-4412-8b0f-ecca9df7be99-kube-api-access-92znt\") pod \"test-curl-7110-140324281430576\" (UID: \"5c30c984-108d-4412-8b0f-ecca9df7be99\") " pod="openshift-ingress/test-curl-7110-140324281430576" Jul 10 19:47:11.874671 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:11.874588 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324281430576" Jul 10 19:47:12.034146 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:12.034107 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324281430576"] Jul 10 19:47:12.037239 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:47:12.037210 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c30c984_108d_4412_8b0f_ecca9df7be99.slice/crio-43f596250b23ae8072bd8e32938aee6fb23fab001a619531a87f80c2efac219b WatchSource:0}: Error finding container 43f596250b23ae8072bd8e32938aee6fb23fab001a619531a87f80c2efac219b: Status 404 returned error can't find the container with id 43f596250b23ae8072bd8e32938aee6fb23fab001a619531a87f80c2efac219b Jul 10 19:47:12.039020 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:12.038997 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:47:12.591825 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:12.591779 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324281430576" event={"ID":"5c30c984-108d-4412-8b0f-ecca9df7be99","Type":"ContainerStarted","Data":"43f596250b23ae8072bd8e32938aee6fb23fab001a619531a87f80c2efac219b"} Jul 10 19:47:13.598843 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:13.598808 2556 generic.go:356] "Generic (PLEG): container finished" podID="5c30c984-108d-4412-8b0f-ecca9df7be99" containerID="c679a505546025e8ea1ef3489e3759f62e708129d9bc9dfbc78d516aa2205039" exitCode=0 Jul 10 19:47:13.599269 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:13.598894 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324281430576" event={"ID":"5c30c984-108d-4412-8b0f-ecca9df7be99","Type":"ContainerDied","Data":"c679a505546025e8ea1ef3489e3759f62e708129d9bc9dfbc78d516aa2205039"} Jul 10 19:47:14.763466 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.763440 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324281430576" Jul 10 19:47:14.806087 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.806055 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7110-140324281430576_5c30c984-108d-4412-8b0f-ecca9df7be99/test-curl-7110-140324281430576/0.log" Jul 10 19:47:14.832496 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.832460 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7110-140324281430576"] Jul 10 19:47:14.842079 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.842047 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7110-140324281430576"] Jul 10 19:47:14.842305 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.842290 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92znt\" (UniqueName: \"kubernetes.io/projected/5c30c984-108d-4412-8b0f-ecca9df7be99-kube-api-access-92znt\") pod \"5c30c984-108d-4412-8b0f-ecca9df7be99\" (UID: \"5c30c984-108d-4412-8b0f-ecca9df7be99\") " Jul 10 19:47:14.844539 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.844511 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c30c984-108d-4412-8b0f-ecca9df7be99-kube-api-access-92znt" (OuterVolumeSpecName: "kube-api-access-92znt") pod "5c30c984-108d-4412-8b0f-ecca9df7be99" (UID: "5c30c984-108d-4412-8b0f-ecca9df7be99"). InnerVolumeSpecName "kube-api-access-92znt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:47:14.944093 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.944008 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92znt\" (UniqueName: \"kubernetes.io/projected/5c30c984-108d-4412-8b0f-ecca9df7be99-kube-api-access-92znt\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:47:14.957996 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.957954 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7110-140324281433776"] Jul 10 19:47:14.958464 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.958447 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c30c984-108d-4412-8b0f-ecca9df7be99" containerName="test-curl-7110-140324281430576" Jul 10 19:47:14.958508 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.958469 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c30c984-108d-4412-8b0f-ecca9df7be99" containerName="test-curl-7110-140324281430576" Jul 10 19:47:14.958594 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.958584 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c30c984-108d-4412-8b0f-ecca9df7be99" containerName="test-curl-7110-140324281430576" Jul 10 19:47:14.961651 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.961627 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324281433776" Jul 10 19:47:14.968491 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:14.968459 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324281433776"] Jul 10 19:47:15.044595 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:15.044551 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69ztp\" (UniqueName: \"kubernetes.io/projected/c2d3f207-ba54-4d1a-941d-d36a4d903f63-kube-api-access-69ztp\") pod \"test-curl-7110-140324281433776\" (UID: \"c2d3f207-ba54-4d1a-941d-d36a4d903f63\") " pod="openshift-ingress/test-curl-7110-140324281433776" Jul 10 19:47:15.145848 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:15.145808 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69ztp\" (UniqueName: \"kubernetes.io/projected/c2d3f207-ba54-4d1a-941d-d36a4d903f63-kube-api-access-69ztp\") pod \"test-curl-7110-140324281433776\" (UID: \"c2d3f207-ba54-4d1a-941d-d36a4d903f63\") " pod="openshift-ingress/test-curl-7110-140324281433776" Jul 10 19:47:15.155274 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:15.155247 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69ztp\" (UniqueName: \"kubernetes.io/projected/c2d3f207-ba54-4d1a-941d-d36a4d903f63-kube-api-access-69ztp\") pod \"test-curl-7110-140324281433776\" (UID: \"c2d3f207-ba54-4d1a-941d-d36a4d903f63\") " pod="openshift-ingress/test-curl-7110-140324281433776" Jul 10 19:47:15.273525 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:15.273483 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324281433776" Jul 10 19:47:15.445267 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:15.445236 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324281433776"] Jul 10 19:47:15.447729 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:47:15.447695 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2d3f207_ba54_4d1a_941d_d36a4d903f63.slice/crio-b9e70437812cc188398358b93b6ce477fdaa7eda735a1cefb06810dcf2428646 WatchSource:0}: Error finding container b9e70437812cc188398358b93b6ce477fdaa7eda735a1cefb06810dcf2428646: Status 404 returned error can't find the container with id b9e70437812cc188398358b93b6ce477fdaa7eda735a1cefb06810dcf2428646 Jul 10 19:47:15.611826 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:15.611719 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324281430576" Jul 10 19:47:15.611826 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:15.611753 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43f596250b23ae8072bd8e32938aee6fb23fab001a619531a87f80c2efac219b" Jul 10 19:47:15.613198 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:15.613170 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324281433776" event={"ID":"c2d3f207-ba54-4d1a-941d-d36a4d903f63","Type":"ContainerStarted","Data":"b9e70437812cc188398358b93b6ce477fdaa7eda735a1cefb06810dcf2428646"} Jul 10 19:47:15.623078 ip-10-0-135-73 kubenswrapper[2556]: E0710 19:47:15.623047 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7110-140324281430576\" is forbidden: User \"system:node:ip-10-0-135-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-73.ec2.internal' and this object" podUID="5c30c984-108d-4412-8b0f-ecca9df7be99" pod="openshift-ingress/test-curl-7110-140324281430576" Jul 10 19:47:16.519848 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:16.519809 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c30c984-108d-4412-8b0f-ecca9df7be99" path="/var/lib/kubelet/pods/5c30c984-108d-4412-8b0f-ecca9df7be99/volumes" Jul 10 19:47:16.621622 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:16.621588 2556 generic.go:356] "Generic (PLEG): container finished" podID="c2d3f207-ba54-4d1a-941d-d36a4d903f63" containerID="a53eeafcdda047228d87029d8df9db0da469785c88b93a6bae0f627dda6d105b" exitCode=0 Jul 10 19:47:16.621817 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:16.621690 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324281433776" event={"ID":"c2d3f207-ba54-4d1a-941d-d36a4d903f63","Type":"ContainerDied","Data":"a53eeafcdda047228d87029d8df9db0da469785c88b93a6bae0f627dda6d105b"} Jul 10 19:47:17.782484 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:17.782457 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324281433776" Jul 10 19:47:17.810139 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:17.810106 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7110-140324281433776_c2d3f207-ba54-4d1a-941d-d36a4d903f63/test-curl-7110-140324281433776/0.log" Jul 10 19:47:17.839195 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:17.839156 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7110-140324281433776"] Jul 10 19:47:17.846600 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:17.846563 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7110-140324281433776"] Jul 10 19:47:17.870999 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:17.870959 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69ztp\" (UniqueName: \"kubernetes.io/projected/c2d3f207-ba54-4d1a-941d-d36a4d903f63-kube-api-access-69ztp\") pod \"c2d3f207-ba54-4d1a-941d-d36a4d903f63\" (UID: \"c2d3f207-ba54-4d1a-941d-d36a4d903f63\") " Jul 10 19:47:17.873257 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:17.873228 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2d3f207-ba54-4d1a-941d-d36a4d903f63-kube-api-access-69ztp" (OuterVolumeSpecName: "kube-api-access-69ztp") pod "c2d3f207-ba54-4d1a-941d-d36a4d903f63" (UID: "c2d3f207-ba54-4d1a-941d-d36a4d903f63"). InnerVolumeSpecName "kube-api-access-69ztp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:47:17.972405 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:17.972357 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69ztp\" (UniqueName: \"kubernetes.io/projected/c2d3f207-ba54-4d1a-941d-d36a4d903f63-kube-api-access-69ztp\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:47:18.003863 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.003821 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:18.004371 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.004351 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d3f207-ba54-4d1a-941d-d36a4d903f63" containerName="test-curl-7110-140324281433776" Jul 10 19:47:18.004472 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.004373 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d3f207-ba54-4d1a-941d-d36a4d903f63" containerName="test-curl-7110-140324281433776" Jul 10 19:47:18.004472 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.004464 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2d3f207-ba54-4d1a-941d-d36a4d903f63" containerName="test-curl-7110-140324281433776" Jul 10 19:47:18.007621 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.007597 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:18.019229 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.019199 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:18.073617 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.073581 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w6qw\" (UniqueName: \"kubernetes.io/projected/809d611b-571b-404d-88fb-f2365586e81d-kube-api-access-7w6qw\") pod \"test-curl-7110-140324280330112\" (UID: \"809d611b-571b-404d-88fb-f2365586e81d\") " pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:18.175064 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.174974 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w6qw\" (UniqueName: \"kubernetes.io/projected/809d611b-571b-404d-88fb-f2365586e81d-kube-api-access-7w6qw\") pod \"test-curl-7110-140324280330112\" (UID: \"809d611b-571b-404d-88fb-f2365586e81d\") " pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:18.184143 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.184112 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w6qw\" (UniqueName: \"kubernetes.io/projected/809d611b-571b-404d-88fb-f2365586e81d-kube-api-access-7w6qw\") pod \"test-curl-7110-140324280330112\" (UID: \"809d611b-571b-404d-88fb-f2365586e81d\") " pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:18.318771 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.318728 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:18.466014 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.465987 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:18.467446 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:47:18.467416 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod809d611b_571b_404d_88fb_f2365586e81d.slice/crio-40c61967db6ed4f337728182cc28b181f4770f82e3f13bed5ccdef3aa52e3ea2 WatchSource:0}: Error finding container 40c61967db6ed4f337728182cc28b181f4770f82e3f13bed5ccdef3aa52e3ea2: Status 404 returned error can't find the container with id 40c61967db6ed4f337728182cc28b181f4770f82e3f13bed5ccdef3aa52e3ea2 Jul 10 19:47:18.518277 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.518243 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2d3f207-ba54-4d1a-941d-d36a4d903f63" path="/var/lib/kubelet/pods/c2d3f207-ba54-4d1a-941d-d36a4d903f63/volumes" Jul 10 19:47:18.636635 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.636593 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324280330112" event={"ID":"809d611b-571b-404d-88fb-f2365586e81d","Type":"ContainerStarted","Data":"40c61967db6ed4f337728182cc28b181f4770f82e3f13bed5ccdef3aa52e3ea2"} Jul 10 19:47:18.637933 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.637909 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324281433776" Jul 10 19:47:18.637933 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:18.637918 2556 scope.go:117] "RemoveContainer" containerID="a53eeafcdda047228d87029d8df9db0da469785c88b93a6bae0f627dda6d105b" Jul 10 19:47:19.645792 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:19.645757 2556 generic.go:356] "Generic (PLEG): container finished" podID="809d611b-571b-404d-88fb-f2365586e81d" containerID="2d96ceb7f61a610b4af3cf5c6dfcf74c8d3b66e0484ab2c52e7e0f963ce2f426" exitCode=0 Jul 10 19:47:19.646257 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:19.645846 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324280330112" event={"ID":"809d611b-571b-404d-88fb-f2365586e81d","Type":"ContainerDied","Data":"2d96ceb7f61a610b4af3cf5c6dfcf74c8d3b66e0484ab2c52e7e0f963ce2f426"} Jul 10 19:47:20.823205 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:20.823171 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:20.846430 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:20.846396 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7110-140324280330112_809d611b-571b-404d-88fb-f2365586e81d/test-curl-7110-140324280330112/0.log" Jul 10 19:47:20.876625 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:20.876588 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:20.879799 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:20.879764 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:20.902496 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:20.902469 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w6qw\" (UniqueName: \"kubernetes.io/projected/809d611b-571b-404d-88fb-f2365586e81d-kube-api-access-7w6qw\") pod \"809d611b-571b-404d-88fb-f2365586e81d\" (UID: \"809d611b-571b-404d-88fb-f2365586e81d\") " Jul 10 19:47:20.904941 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:20.904861 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/809d611b-571b-404d-88fb-f2365586e81d-kube-api-access-7w6qw" (OuterVolumeSpecName: "kube-api-access-7w6qw") pod "809d611b-571b-404d-88fb-f2365586e81d" (UID: "809d611b-571b-404d-88fb-f2365586e81d"). InnerVolumeSpecName "kube-api-access-7w6qw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:47:21.002738 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.002696 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:21.003257 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.003238 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="809d611b-571b-404d-88fb-f2365586e81d" containerName="test-curl-7110-140324280330112" Jul 10 19:47:21.003325 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.003261 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="809d611b-571b-404d-88fb-f2365586e81d" containerName="test-curl-7110-140324280330112" Jul 10 19:47:21.003392 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.003381 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="809d611b-571b-404d-88fb-f2365586e81d" containerName="test-curl-7110-140324280330112" Jul 10 19:47:21.003516 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.003494 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w6qw\" (UniqueName: \"kubernetes.io/projected/809d611b-571b-404d-88fb-f2365586e81d-kube-api-access-7w6qw\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:47:21.006515 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.006495 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:21.016542 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.016512 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:21.104751 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.104709 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28dp5\" (UniqueName: \"kubernetes.io/projected/7a1f13f5-88d2-463c-b6d6-3497820428fe-kube-api-access-28dp5\") pod \"test-curl-7110-140324280330112\" (UID: \"7a1f13f5-88d2-463c-b6d6-3497820428fe\") " pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:21.205593 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.205497 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28dp5\" (UniqueName: \"kubernetes.io/projected/7a1f13f5-88d2-463c-b6d6-3497820428fe-kube-api-access-28dp5\") pod \"test-curl-7110-140324280330112\" (UID: \"7a1f13f5-88d2-463c-b6d6-3497820428fe\") " pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:21.214784 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.214746 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28dp5\" (UniqueName: \"kubernetes.io/projected/7a1f13f5-88d2-463c-b6d6-3497820428fe-kube-api-access-28dp5\") pod \"test-curl-7110-140324280330112\" (UID: \"7a1f13f5-88d2-463c-b6d6-3497820428fe\") " pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:21.316841 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.316794 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:21.469351 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.469322 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:21.470851 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:47:21.470826 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a1f13f5_88d2_463c_b6d6_3497820428fe.slice/crio-8b5930cca3308e025937c75a7e7dcb6d98ca800c9f1acf1db6ba5cfdd1d122ef WatchSource:0}: Error finding container 8b5930cca3308e025937c75a7e7dcb6d98ca800c9f1acf1db6ba5cfdd1d122ef: Status 404 returned error can't find the container with id 8b5930cca3308e025937c75a7e7dcb6d98ca800c9f1acf1db6ba5cfdd1d122ef Jul 10 19:47:21.661933 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.661894 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324280330112" event={"ID":"7a1f13f5-88d2-463c-b6d6-3497820428fe","Type":"ContainerStarted","Data":"8b5930cca3308e025937c75a7e7dcb6d98ca800c9f1acf1db6ba5cfdd1d122ef"} Jul 10 19:47:21.663248 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.663224 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40c61967db6ed4f337728182cc28b181f4770f82e3f13bed5ccdef3aa52e3ea2" Jul 10 19:47:21.663248 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.663247 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:21.675412 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:21.675375 2556 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7110-140324280330112" oldPodUID="809d611b-571b-404d-88fb-f2365586e81d" podUID="7a1f13f5-88d2-463c-b6d6-3497820428fe" Jul 10 19:47:22.518683 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:22.518645 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="809d611b-571b-404d-88fb-f2365586e81d" path="/var/lib/kubelet/pods/809d611b-571b-404d-88fb-f2365586e81d/volumes" Jul 10 19:47:22.670925 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:22.670886 2556 generic.go:356] "Generic (PLEG): container finished" podID="7a1f13f5-88d2-463c-b6d6-3497820428fe" containerID="4889850155d2b3519ef4df38c38db483a9492320742491ccc593c9c74ddce7bd" exitCode=0 Jul 10 19:47:22.671117 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:22.670925 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324280330112" event={"ID":"7a1f13f5-88d2-463c-b6d6-3497820428fe","Type":"ContainerDied","Data":"4889850155d2b3519ef4df38c38db483a9492320742491ccc593c9c74ddce7bd"} Jul 10 19:47:23.830230 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:23.830202 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:23.850915 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:23.850847 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7110-140324280330112_7a1f13f5-88d2-463c-b6d6-3497820428fe/test-curl-7110-140324280330112/0.log" Jul 10 19:47:23.877187 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:23.877149 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:23.881213 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:23.881183 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7110-140324280330112"] Jul 10 19:47:23.931170 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:23.931086 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28dp5\" (UniqueName: \"kubernetes.io/projected/7a1f13f5-88d2-463c-b6d6-3497820428fe-kube-api-access-28dp5\") pod \"7a1f13f5-88d2-463c-b6d6-3497820428fe\" (UID: \"7a1f13f5-88d2-463c-b6d6-3497820428fe\") " Jul 10 19:47:23.933341 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:23.933314 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a1f13f5-88d2-463c-b6d6-3497820428fe-kube-api-access-28dp5" (OuterVolumeSpecName: "kube-api-access-28dp5") pod "7a1f13f5-88d2-463c-b6d6-3497820428fe" (UID: "7a1f13f5-88d2-463c-b6d6-3497820428fe"). InnerVolumeSpecName "kube-api-access-28dp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:47:24.003347 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.003308 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:24.003752 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.003738 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a1f13f5-88d2-463c-b6d6-3497820428fe" containerName="test-curl-7110-140324280330112" Jul 10 19:47:24.003801 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.003753 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a1f13f5-88d2-463c-b6d6-3497820428fe" containerName="test-curl-7110-140324280330112" Jul 10 19:47:24.003841 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.003836 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a1f13f5-88d2-463c-b6d6-3497820428fe" containerName="test-curl-7110-140324280330112" Jul 10 19:47:24.006750 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.006731 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:24.015180 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.015141 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:24.032108 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.032073 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28dp5\" (UniqueName: \"kubernetes.io/projected/7a1f13f5-88d2-463c-b6d6-3497820428fe-kube-api-access-28dp5\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:47:24.132756 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.132710 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk6rf\" (UniqueName: \"kubernetes.io/projected/5a330e59-1973-4984-9d4c-e5a060964579-kube-api-access-pk6rf\") pod \"test-curl-7110-140324280331120\" (UID: \"5a330e59-1973-4984-9d4c-e5a060964579\") " pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:24.233400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.233307 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk6rf\" (UniqueName: \"kubernetes.io/projected/5a330e59-1973-4984-9d4c-e5a060964579-kube-api-access-pk6rf\") pod \"test-curl-7110-140324280331120\" (UID: \"5a330e59-1973-4984-9d4c-e5a060964579\") " pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:24.244657 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.244618 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk6rf\" (UniqueName: \"kubernetes.io/projected/5a330e59-1973-4984-9d4c-e5a060964579-kube-api-access-pk6rf\") pod \"test-curl-7110-140324280331120\" (UID: \"5a330e59-1973-4984-9d4c-e5a060964579\") " pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:24.317631 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.317593 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:24.466020 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.465991 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:24.467285 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:47:24.467256 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a330e59_1973_4984_9d4c_e5a060964579.slice/crio-b34c9d648a50a712cdbb19cd1f9ba3d79ae74c3ece9ee0a637e50b7ce265c962 WatchSource:0}: Error finding container b34c9d648a50a712cdbb19cd1f9ba3d79ae74c3ece9ee0a637e50b7ce265c962: Status 404 returned error can't find the container with id b34c9d648a50a712cdbb19cd1f9ba3d79ae74c3ece9ee0a637e50b7ce265c962 Jul 10 19:47:24.523329 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.523282 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a1f13f5-88d2-463c-b6d6-3497820428fe" path="/var/lib/kubelet/pods/7a1f13f5-88d2-463c-b6d6-3497820428fe/volumes" Jul 10 19:47:24.684528 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.684492 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324280331120" event={"ID":"5a330e59-1973-4984-9d4c-e5a060964579","Type":"ContainerStarted","Data":"b34c9d648a50a712cdbb19cd1f9ba3d79ae74c3ece9ee0a637e50b7ce265c962"} Jul 10 19:47:24.685938 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.685912 2556 scope.go:117] "RemoveContainer" containerID="4889850155d2b3519ef4df38c38db483a9492320742491ccc593c9c74ddce7bd" Jul 10 19:47:24.686074 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:24.685913 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280330112" Jul 10 19:47:25.693376 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:25.693342 2556 generic.go:356] "Generic (PLEG): container finished" podID="5a330e59-1973-4984-9d4c-e5a060964579" containerID="ade17a77ad7899f5ad1252460e403ae6001e1cbb7dcff78840643419bf296c07" exitCode=0 Jul 10 19:47:25.694100 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:25.693425 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324280331120" event={"ID":"5a330e59-1973-4984-9d4c-e5a060964579","Type":"ContainerDied","Data":"ade17a77ad7899f5ad1252460e403ae6001e1cbb7dcff78840643419bf296c07"} Jul 10 19:47:26.858514 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:26.858477 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:26.883383 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:26.883343 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7110-140324280331120_5a330e59-1973-4984-9d4c-e5a060964579/test-curl-7110-140324280331120/0.log" Jul 10 19:47:26.910917 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:26.910852 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:26.915802 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:26.915765 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:26.959620 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:26.959522 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk6rf\" (UniqueName: \"kubernetes.io/projected/5a330e59-1973-4984-9d4c-e5a060964579-kube-api-access-pk6rf\") pod \"5a330e59-1973-4984-9d4c-e5a060964579\" (UID: \"5a330e59-1973-4984-9d4c-e5a060964579\") " Jul 10 19:47:26.961886 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:26.961835 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a330e59-1973-4984-9d4c-e5a060964579-kube-api-access-pk6rf" (OuterVolumeSpecName: "kube-api-access-pk6rf") pod "5a330e59-1973-4984-9d4c-e5a060964579" (UID: "5a330e59-1973-4984-9d4c-e5a060964579"). InnerVolumeSpecName "kube-api-access-pk6rf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:47:27.046350 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.046306 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:27.046777 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.046761 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a330e59-1973-4984-9d4c-e5a060964579" containerName="test-curl-7110-140324280331120" Jul 10 19:47:27.046822 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.046779 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a330e59-1973-4984-9d4c-e5a060964579" containerName="test-curl-7110-140324280331120" Jul 10 19:47:27.046894 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.046866 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a330e59-1973-4984-9d4c-e5a060964579" containerName="test-curl-7110-140324280331120" Jul 10 19:47:27.050106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.050086 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:27.056505 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.056462 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:27.060534 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.060503 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk6rf\" (UniqueName: \"kubernetes.io/projected/5a330e59-1973-4984-9d4c-e5a060964579-kube-api-access-pk6rf\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:47:27.161186 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.161146 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvlsg\" (UniqueName: \"kubernetes.io/projected/b2a17212-72cb-4f8f-9fbe-ad08870300ab-kube-api-access-cvlsg\") pod \"test-curl-7110-140324280331120\" (UID: \"b2a17212-72cb-4f8f-9fbe-ad08870300ab\") " pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:27.262483 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.262438 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlsg\" (UniqueName: \"kubernetes.io/projected/b2a17212-72cb-4f8f-9fbe-ad08870300ab-kube-api-access-cvlsg\") pod \"test-curl-7110-140324280331120\" (UID: \"b2a17212-72cb-4f8f-9fbe-ad08870300ab\") " pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:27.274261 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.274220 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvlsg\" (UniqueName: \"kubernetes.io/projected/b2a17212-72cb-4f8f-9fbe-ad08870300ab-kube-api-access-cvlsg\") pod \"test-curl-7110-140324280331120\" (UID: \"b2a17212-72cb-4f8f-9fbe-ad08870300ab\") " pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:27.361377 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.361332 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:27.507988 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.507959 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:27.510215 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:47:27.510186 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2a17212_72cb_4f8f_9fbe_ad08870300ab.slice/crio-f7cf8fb507470ed3213b3531f495b10bb7f92e2d34d66b3e004db8ac922b1059 WatchSource:0}: Error finding container f7cf8fb507470ed3213b3531f495b10bb7f92e2d34d66b3e004db8ac922b1059: Status 404 returned error can't find the container with id f7cf8fb507470ed3213b3531f495b10bb7f92e2d34d66b3e004db8ac922b1059 Jul 10 19:47:27.708728 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.708703 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:27.708923 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.708749 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b34c9d648a50a712cdbb19cd1f9ba3d79ae74c3ece9ee0a637e50b7ce265c962" Jul 10 19:47:27.710151 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.710117 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324280331120" event={"ID":"b2a17212-72cb-4f8f-9fbe-ad08870300ab","Type":"ContainerStarted","Data":"f7cf8fb507470ed3213b3531f495b10bb7f92e2d34d66b3e004db8ac922b1059"} Jul 10 19:47:27.720608 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:27.720577 2556 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7110-140324280331120" oldPodUID="5a330e59-1973-4984-9d4c-e5a060964579" podUID="b2a17212-72cb-4f8f-9fbe-ad08870300ab" Jul 10 19:47:28.518499 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:28.518464 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a330e59-1973-4984-9d4c-e5a060964579" path="/var/lib/kubelet/pods/5a330e59-1973-4984-9d4c-e5a060964579/volumes" Jul 10 19:47:28.689958 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:28.689925 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:47:28.691410 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:28.691386 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:47:28.693179 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:28.693157 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:47:28.694528 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:28.694504 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-29hgv_14e7d49a-c59a-41c4-b0e7-357cc681a3a8/ovn-acl-logging/0.log" Jul 10 19:47:28.717845 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:28.717807 2556 generic.go:356] "Generic (PLEG): container finished" podID="b2a17212-72cb-4f8f-9fbe-ad08870300ab" containerID="58ad62cbbfff6f95e4209cc1b42d5c0e1f5812fec1516d939acd0d6e9f7c215f" exitCode=0 Jul 10 19:47:28.718051 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:28.717903 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7110-140324280331120" event={"ID":"b2a17212-72cb-4f8f-9fbe-ad08870300ab","Type":"ContainerDied","Data":"58ad62cbbfff6f95e4209cc1b42d5c0e1f5812fec1516d939acd0d6e9f7c215f"} Jul 10 19:47:29.879455 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:29.879426 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:29.903419 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:29.903380 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7110-140324280331120_b2a17212-72cb-4f8f-9fbe-ad08870300ab/test-curl-7110-140324280331120/0.log" Jul 10 19:47:29.926567 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:29.926536 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:29.929014 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:29.928992 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7110-140324280331120"] Jul 10 19:47:29.988629 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:29.988594 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvlsg\" (UniqueName: \"kubernetes.io/projected/b2a17212-72cb-4f8f-9fbe-ad08870300ab-kube-api-access-cvlsg\") pod \"b2a17212-72cb-4f8f-9fbe-ad08870300ab\" (UID: \"b2a17212-72cb-4f8f-9fbe-ad08870300ab\") " Jul 10 19:47:29.990948 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:29.990921 2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2a17212-72cb-4f8f-9fbe-ad08870300ab-kube-api-access-cvlsg" (OuterVolumeSpecName: "kube-api-access-cvlsg") pod "b2a17212-72cb-4f8f-9fbe-ad08870300ab" (UID: "b2a17212-72cb-4f8f-9fbe-ad08870300ab"). InnerVolumeSpecName "kube-api-access-cvlsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 19:47:30.089612 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:30.089569 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvlsg\" (UniqueName: \"kubernetes.io/projected/b2a17212-72cb-4f8f-9fbe-ad08870300ab-kube-api-access-cvlsg\") on node \"ip-10-0-135-73.ec2.internal\" DevicePath \"\"" Jul 10 19:47:30.518545 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:30.518509 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2a17212-72cb-4f8f-9fbe-ad08870300ab" path="/var/lib/kubelet/pods/b2a17212-72cb-4f8f-9fbe-ad08870300ab/volumes" Jul 10 19:47:30.732520 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:30.732491 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7110-140324280331120" Jul 10 19:47:30.732520 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:30.732506 2556 scope.go:117] "RemoveContainer" containerID="58ad62cbbfff6f95e4209cc1b42d5c0e1f5812fec1516d939acd0d6e9f7c215f" Jul 10 19:47:45.322674 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:45.322633 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55b5d99cb5-lbhbc_7dc4ad0b-63ec-47e6-bfb0-eace2ed50998/authorino/0.log" Jul 10 19:47:50.444044 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:50.444007 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8r9zh_3b63c270-508e-4943-82f3-61a1feae75fd/manager/0.log" Jul 10 19:47:50.575733 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:50.575695 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qs4zr_69647bae-3f04-4178-a377-8a8f34dd2fdc/manager/0.log" Jul 10 19:47:50.705090 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:50.704996 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6886c467c5-6x9pm_a8f0c329-6f7f-4427-b1ff-d127b84d1302/manager/0.log" Jul 10 19:47:50.821172 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:50.821143 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hf7fn_be88e0b7-b17f-46bb-a160-fe4f7f3294f9/server/0.log" Jul 10 19:47:50.955601 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:50.955519 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xbsqz_c5f4c650-01f9-4513-9951-cf1637e933be/manager/0.log" Jul 10 19:47:51.356839 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:51.356811 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d686dc7f-vdpbp_b5b67215-f589-4689-b233-2179378b9690/copy-manifests/0.log" Jul 10 19:47:51.375436 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:51.375399 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d686dc7f-vdpbp_b5b67215-f589-4689-b233-2179378b9690/manager/0.log" Jul 10 19:47:52.725575 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:52.725542 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55b5d99cb5-lbhbc_7dc4ad0b-63ec-47e6-bfb0-eace2ed50998/authorino/0.log" Jul 10 19:47:52.861541 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:52.861489 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-558c6fb9c4-gtb4k_a420d721-86ed-48b9-9eed-e69e2396256b/manager/0.log" Jul 10 19:47:52.984824 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:52.984733 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5546559d7-d5rlf_c3badc99-0410-4cf2-8044-19de0ae1be89/manager/0.log" Jul 10 19:47:53.241046 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:53.241010 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-7569bc7c69-sz6qf_c480bb25-68fa-4da6-b963-929728e4576a/manager/0.log" Jul 10 19:47:53.495079 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:53.494995 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-7d6bb9cf99-qrp4p_65ca1260-f825-4690-95f8-4663cf06f13c/manager/0.log" Jul 10 19:47:53.888169 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:53.888093 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gc88f_00e32cf7-d620-4529-b1f9-a77357c329ef/istio-proxy/0.log" Jul 10 19:47:54.263132 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:54.263105 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-967644b46-jmc4g_901caad2-0055-499e-a5d5-77e23ed88deb/kube-auth-proxy/0.log" Jul 10 19:47:54.387782 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:54.387750 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sc64n_821b101f-507a-422a-83c3-58207b76e691/istio-proxy/1.log" Jul 10 19:47:54.511144 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:54.511111 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mnfxk_b6a7004c-8886-4a50-a427-c64ef9aec704/payload-pre-processing/0.log" Jul 10 19:47:54.760014 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:54.759988 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c65fc6986-nkwgn_1510cc81-e171-470a-bc51-ab3243a880b7/router/0.log" Jul 10 19:47:55.134756 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:55.134673 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-btkm7_287918a2-0013-4a2f-9431-f0d0dba40b92/main/0.log" Jul 10 19:47:55.380394 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:55.380364 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2dhhh_fd6a4418-4caa-4105-bf9b-f37037618989/main/0.log" Jul 10 19:47:55.750411 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:47:55.750375 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-t22r4_4e19fc92-05d3-453c-9165-557b9e4fc75d/main/0.log" Jul 10 19:48:02.806257 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:02.806219 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h42d9_9c4bb811-9e79-4b09-84fe-a6e7debe4cee/global-pull-secret-syncer/0.log" Jul 10 19:48:03.002070 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:03.002034 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fqzkj_da9b854d-8ef1-4986-841d-25e45fd65bab/konnectivity-agent/0.log" Jul 10 19:48:03.092056 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:03.091959 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-73.ec2.internal_24c9a12d95ac9c34c371cd5589b6402d/haproxy/0.log" Jul 10 19:48:07.547242 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:07.547202 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55b5d99cb5-lbhbc_7dc4ad0b-63ec-47e6-bfb0-eace2ed50998/authorino/0.log" Jul 10 19:48:07.583004 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:07.582969 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-558c6fb9c4-gtb4k_a420d721-86ed-48b9-9eed-e69e2396256b/manager/0.log" Jul 10 19:48:07.619657 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:07.619622 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5546559d7-d5rlf_c3badc99-0410-4cf2-8044-19de0ae1be89/manager/0.log" Jul 10 19:48:07.856816 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:07.856726 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-7569bc7c69-sz6qf_c480bb25-68fa-4da6-b963-929728e4576a/manager/0.log" Jul 10 19:48:08.006026 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:08.005986 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-7d6bb9cf99-qrp4p_65ca1260-f825-4690-95f8-4663cf06f13c/manager/0.log" Jul 10 19:48:09.696708 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:09.696662 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-bn47g_dae262dc-c996-4f1d-b118-75619fbd3609/cluster-monitoring-operator/0.log" Jul 10 19:48:09.737457 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:09.737431 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-2h5dm_3936321e-af64-4cdc-939a-db6f017e7459/kube-state-metrics/0.log" Jul 10 19:48:09.758517 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:09.758476 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-2h5dm_3936321e-af64-4cdc-939a-db6f017e7459/kube-rbac-proxy-main/0.log" Jul 10 19:48:09.785071 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:09.785010 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-2h5dm_3936321e-af64-4cdc-939a-db6f017e7459/kube-rbac-proxy-self/0.log" Jul 10 19:48:10.064289 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:10.064203 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-797tj_8c72933f-91a8-434e-9984-9fce7ef47035/node-exporter/0.log" Jul 10 19:48:10.096188 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:10.096135 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-797tj_8c72933f-91a8-434e-9984-9fce7ef47035/kube-rbac-proxy/0.log" Jul 10 19:48:10.123278 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:10.123252 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-797tj_8c72933f-91a8-434e-9984-9fce7ef47035/init-textfile/0.log" Jul 10 19:48:12.346834 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.346798 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t"] Jul 10 19:48:12.347287 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.347269 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2a17212-72cb-4f8f-9fbe-ad08870300ab" containerName="test-curl-7110-140324280331120" Jul 10 19:48:12.347333 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.347289 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a17212-72cb-4f8f-9fbe-ad08870300ab" containerName="test-curl-7110-140324280331120" Jul 10 19:48:12.347386 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.347376 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2a17212-72cb-4f8f-9fbe-ad08870300ab" containerName="test-curl-7110-140324280331120" Jul 10 19:48:12.350551 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.350530 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.352790 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.352768 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kffvc\"/\"default-dockercfg-c6tm2\"" Jul 10 19:48:12.353747 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.353727 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kffvc\"/\"kube-root-ca.crt\"" Jul 10 19:48:12.353895 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.353809 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kffvc\"/\"openshift-service-ca.crt\"" Jul 10 19:48:12.358604 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.358556 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t"] Jul 10 19:48:12.483015 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.482971 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-sys\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.483204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.483025 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldjlb\" (UniqueName: \"kubernetes.io/projected/d246bad2-5d4d-4a97-9e52-74f734101877-kube-api-access-ldjlb\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.483204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.483082 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-lib-modules\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.483204 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.483109 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-proc\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.483311 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.483221 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-podres\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.584106 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.584062 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-lib-modules\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.584298 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.584112 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-proc\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.584298 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.584166 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-podres\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.584298 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.584214 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-sys\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.584298 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.584249 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldjlb\" (UniqueName: \"kubernetes.io/projected/d246bad2-5d4d-4a97-9e52-74f734101877-kube-api-access-ldjlb\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.584298 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.584257 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-lib-modules\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.584298 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.584264 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-proc\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.584519 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.584310 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-sys\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.584519 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.584358 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d246bad2-5d4d-4a97-9e52-74f734101877-podres\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.594900 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.594853 2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldjlb\" (UniqueName: \"kubernetes.io/projected/d246bad2-5d4d-4a97-9e52-74f734101877-kube-api-access-ldjlb\") pod \"perf-node-gather-daemonset-76b2t\" (UID: \"d246bad2-5d4d-4a97-9e52-74f734101877\") " pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.663032 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.662944 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:12.731790 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.731729 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/1.log" Jul 10 19:48:12.746091 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.746056 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-gxdsf_633078d5-6344-4d3c-b710-f205326c20f5/console-operator/2.log" Jul 10 19:48:12.824481 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.824440 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t"] Jul 10 19:48:12.825866 ip-10-0-135-73 kubenswrapper[2556]: W0710 19:48:12.825826 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd246bad2_5d4d_4a97_9e52_74f734101877.slice/crio-b62e5be642b610569b946118ddedfe83668d3164621d3de887de03bedb3cd57b WatchSource:0}: Error finding container b62e5be642b610569b946118ddedfe83668d3164621d3de887de03bedb3cd57b: Status 404 returned error can't find the container with id b62e5be642b610569b946118ddedfe83668d3164621d3de887de03bedb3cd57b Jul 10 19:48:12.998423 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.998380 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" event={"ID":"d246bad2-5d4d-4a97-9e52-74f734101877","Type":"ContainerStarted","Data":"b00200c64eed3088e2c12312b724b051e2f45e5e99c43f72b3d17a83a3d35393"} Jul 10 19:48:12.998653 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.998445 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" event={"ID":"d246bad2-5d4d-4a97-9e52-74f734101877","Type":"ContainerStarted","Data":"b62e5be642b610569b946118ddedfe83668d3164621d3de887de03bedb3cd57b"} Jul 10 19:48:12.998653 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:12.998508 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:13.029299 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:13.029244 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" podStartSLOduration=1.029225349 podStartE2EDuration="1.029225349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:48:13.027833623 +0000 UTC m=+2445.044291991" watchObservedRunningTime="2026-07-10 19:48:13.029225349 +0000 UTC m=+2445.045683719" Jul 10 19:48:14.055834 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:14.055797 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-n2fff_230092b8-b633-4800-8567-7b7064570b72/volume-data-source-validator/0.log" Jul 10 19:48:15.210231 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:15.210194 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fqbbz_9125148c-8264-4c42-bb5a-a811fc4f5588/dns/0.log" Jul 10 19:48:15.237432 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:15.237402 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fqbbz_9125148c-8264-4c42-bb5a-a811fc4f5588/kube-rbac-proxy/0.log" Jul 10 19:48:15.320269 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:15.320237 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ctm9t_669e056e-e810-451c-a00f-4e1c0a9d4c26/dns-node-resolver/0.log" Jul 10 19:48:15.953241 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:15.953206 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p67tg_46f8b6c0-73ab-4684-91c6-04e358626759/node-ca/0.log" Jul 10 19:48:16.881507 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:16.881472 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gc88f_00e32cf7-d620-4529-b1f9-a77357c329ef/istio-proxy/0.log" Jul 10 19:48:17.059794 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:17.059761 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-967644b46-jmc4g_901caad2-0055-499e-a5d5-77e23ed88deb/kube-auth-proxy/0.log" Jul 10 19:48:17.095024 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:17.094985 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sc64n_821b101f-507a-422a-83c3-58207b76e691/istio-proxy/0.log" Jul 10 19:48:17.109631 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:17.109597 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sc64n_821b101f-507a-422a-83c3-58207b76e691/istio-proxy/1.log" Jul 10 19:48:17.133577 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:17.133492 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mnfxk_b6a7004c-8886-4a50-a427-c64ef9aec704/payload-pre-processing/0.log" Jul 10 19:48:17.190007 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:17.189968 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c65fc6986-nkwgn_1510cc81-e171-470a-bc51-ab3243a880b7/router/0.log" Jul 10 19:48:17.925375 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:17.925331 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pgqt4_1045b744-a760-41ab-9e4d-ffe3ad4f80fb/serve-healthcheck-canary/0.log" Jul 10 19:48:18.424746 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:18.424705 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-gqxwz_324a3506-415f-46fc-94f0-47029379437d/insights-operator/0.log" Jul 10 19:48:18.426030 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:18.426003 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-gqxwz_324a3506-415f-46fc-94f0-47029379437d/insights-operator/1.log" Jul 10 19:48:18.609107 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:18.609077 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l7g8z_c36cf591-5252-4695-8047-20f5a08d5908/kube-rbac-proxy/0.log" Jul 10 19:48:18.636350 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:18.636313 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l7g8z_c36cf591-5252-4695-8047-20f5a08d5908/exporter/0.log" Jul 10 19:48:18.665067 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:18.665035 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l7g8z_c36cf591-5252-4695-8047-20f5a08d5908/extractor/0.log" Jul 10 19:48:19.035614 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:19.035580 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kffvc/perf-node-gather-daemonset-76b2t" Jul 10 19:48:20.838400 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:20.838367 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8r9zh_3b63c270-508e-4943-82f3-61a1feae75fd/manager/0.log" Jul 10 19:48:21.088189 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:21.088158 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qs4zr_69647bae-3f04-4178-a377-8a8f34dd2fdc/manager/0.log" Jul 10 19:48:21.259599 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:21.259561 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6886c467c5-6x9pm_a8f0c329-6f7f-4427-b1ff-d127b84d1302/manager/0.log" Jul 10 19:48:21.280001 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:21.279964 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hf7fn_be88e0b7-b17f-46bb-a160-fe4f7f3294f9/server/0.log" Jul 10 19:48:21.398574 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:21.398537 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xbsqz_c5f4c650-01f9-4513-9951-cf1637e933be/manager/0.log" Jul 10 19:48:21.657533 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:21.657486 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d686dc7f-vdpbp_b5b67215-f589-4689-b233-2179378b9690/manager/0.log" Jul 10 19:48:21.678922 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:21.678857 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d686dc7f-vdpbp_b5b67215-f589-4689-b233-2179378b9690/copy-manifests/0.log" Jul 10 19:48:22.905645 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:22.905617 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gc88f_00e32cf7-d620-4529-b1f9-a77357c329ef/istio-proxy/0.log" Jul 10 19:48:23.057688 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:23.057653 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-967644b46-jmc4g_901caad2-0055-499e-a5d5-77e23ed88deb/kube-auth-proxy/0.log" Jul 10 19:48:23.084536 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:23.084505 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sc64n_821b101f-507a-422a-83c3-58207b76e691/istio-proxy/0.log" Jul 10 19:48:23.098318 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:23.098282 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sc64n_821b101f-507a-422a-83c3-58207b76e691/istio-proxy/1.log" Jul 10 19:48:23.112572 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:23.112540 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mnfxk_b6a7004c-8886-4a50-a427-c64ef9aec704/payload-pre-processing/0.log" Jul 10 19:48:23.143999 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:23.143965 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c65fc6986-nkwgn_1510cc81-e171-470a-bc51-ab3243a880b7/router/0.log" Jul 10 19:48:24.345282 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:24.345252 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gc88f_00e32cf7-d620-4529-b1f9-a77357c329ef/istio-proxy/0.log" Jul 10 19:48:24.506304 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:24.506267 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-967644b46-jmc4g_901caad2-0055-499e-a5d5-77e23ed88deb/kube-auth-proxy/0.log" Jul 10 19:48:24.529712 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:24.529676 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sc64n_821b101f-507a-422a-83c3-58207b76e691/istio-proxy/0.log" Jul 10 19:48:24.544122 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:24.544080 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sc64n_821b101f-507a-422a-83c3-58207b76e691/istio-proxy/1.log" Jul 10 19:48:24.558441 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:24.558417 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-mnfxk_b6a7004c-8886-4a50-a427-c64ef9aec704/payload-pre-processing/0.log" Jul 10 19:48:24.598252 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:24.598225 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c65fc6986-nkwgn_1510cc81-e171-470a-bc51-ab3243a880b7/router/0.log" Jul 10 19:48:27.933460 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:27.933433 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-jhpkc_3614c216-09cd-4b68-bdd9-42286a6c2454/migrator/0.log" Jul 10 19:48:27.976448 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:27.976411 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-jhpkc_3614c216-09cd-4b68-bdd9-42286a6c2454/graceful-termination/0.log" Jul 10 19:48:28.388814 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:28.388784 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-pzvv2_65b055a9-5e50-44ee-9e0a-000735040140/kube-storage-version-migrator-operator/1.log" Jul 10 19:48:28.390481 ip-10-0-135-73 kubenswrapper[2556]: I0710 19:48:28.390460 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-pzvv2_65b055a9-5e50-44ee-9e0a-000735040140/kube-storage-version-migrator-operator/0.log"