Jul 22 20:01:30.319752 ip-10-0-139-68 systemd[1]: Starting Kubernetes Kubelet... Jul 22 20:01:30.738866 ip-10-0-139-68 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 20:01:30.738866 ip-10-0-139-68 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 20:01:30.738866 ip-10-0-139-68 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 20:01:30.738866 ip-10-0-139-68 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 20:01:30.738866 ip-10-0-139-68 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 20:01:30.740861 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.740719 2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747446 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747472 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747475 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747479 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747482 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747485 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747489 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747494 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747497 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747500 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 20:01:30.747483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747503 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747507 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747510 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747513 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747516 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747520 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747522 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747525 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747527 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747530 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747533 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747535 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747538 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747544 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747547 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747551 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747553 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747556 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747558 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747561 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 20:01:30.747893 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747563 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747566 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747568 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747571 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747573 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747578 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747581 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747583 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747586 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747589 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747592 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747594 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747597 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747600 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747603 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747606 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747609 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747611 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747614 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747617 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 20:01:30.748363 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747619 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747623 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747625 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747635 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747639 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747641 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747644 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747648 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747653 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747656 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747659 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747663 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747667 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747670 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747673 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747676 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747680 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747685 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747689 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 20:01:30.748872 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747691 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747694 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747697 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747700 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747703 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747705 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747708 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747711 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747714 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747716 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747719 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747722 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747724 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747727 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747731 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747734 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747737 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747740 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747742 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747744 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 20:01:30.749375 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747747 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747749 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747751 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747754 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747757 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747759 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747761 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747764 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747767 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.747770 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748602 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748612 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748615 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748619 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748622 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748626 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748630 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748633 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748636 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748639 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 20:01:30.749946 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748642 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748644 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748647 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748650 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748653 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748657 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748661 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748665 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748668 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748671 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748674 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748677 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748679 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748682 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748684 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748687 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748690 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748693 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748697 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748700 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 20:01:30.750483 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748703 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748706 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748709 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748711 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748714 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748717 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748720 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748722 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748726 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748728 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748731 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748734 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748736 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748740 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748742 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748745 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748748 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748751 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748753 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748756 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 20:01:30.750994 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748758 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748761 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748764 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748766 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748769 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748771 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748774 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748776 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748779 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748782 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748784 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748787 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748793 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748796 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748798 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748801 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748803 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748807 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748810 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 20:01:30.751514 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748812 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748815 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748818 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748820 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748823 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748825 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748828 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748831 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748837 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748840 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748843 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748845 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748848 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748851 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748856 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748859 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748861 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748864 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748867 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 20:01:30.751986 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748869 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748872 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748875 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748878 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748880 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748883 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748888 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748890 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748893 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748895 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.748898 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749328 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749333 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749336 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749339 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749342 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749344 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749347 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749350 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749352 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 20:01:30.752473 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749355 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749360 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749364 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749367 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749370 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749373 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749376 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749380 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749383 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749386 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749389 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749392 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749395 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749398 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749400 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749403 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749406 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749427 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749434 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749437 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 20:01:30.752963 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749439 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749444 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749448 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749451 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749454 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749457 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749461 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749463 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749466 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749469 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749471 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749474 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749477 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749479 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749482 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749484 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749486 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749489 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749492 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 20:01:30.753471 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749495 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749499 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749502 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749505 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749508 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749510 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749513 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749515 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749518 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749521 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749524 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749528 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749531 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749533 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749536 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749538 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749541 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749543 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749546 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749549 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 20:01:30.753960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749553 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749556 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749558 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749561 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749563 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749566 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749570 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749573 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749576 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749597 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749600 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749603 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749606 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749609 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749614 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749617 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749620 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749623 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749626 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 20:01:30.754553 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749629 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749632 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749634 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749645 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749651 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749654 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749657 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749659 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749662 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749665 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749667 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.749670 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749777 2553 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749792 2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749799 2553 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749804 2553 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749809 2553 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749812 2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749816 2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749821 2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749824 2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 20:01:30.755052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749828 2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749832 2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749835 2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749839 2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749842 2553 flags.go:64] FLAG: --cgroup-root="" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749845 2553 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749848 2553 flags.go:64] FLAG: --client-ca-file="" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749852 2553 flags.go:64] FLAG: --cloud-provider="external" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749855 2553 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749860 2553 flags.go:64] FLAG: --cluster-domain="" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749862 2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749866 2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749869 2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749872 2553 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749876 2553 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749881 2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749884 2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749888 2553 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749891 2553 flags.go:64] FLAG: --contention-profiling="false" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749894 2553 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749897 2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749901 2553 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749904 2553 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749908 2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 20:01:30.755726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749911 2553 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749915 2553 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749918 2553 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749921 2553 flags.go:64] FLAG: --enable-server="true" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749924 2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749929 2553 flags.go:64] FLAG: --event-burst="100" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749932 2553 flags.go:64] FLAG: --event-qps="50" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749934 2553 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749938 2553 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749941 2553 flags.go:64] FLAG: --eviction-hard="" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749946 2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749949 2553 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749952 2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749955 2553 flags.go:64] FLAG: --eviction-soft="" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749959 2553 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749962 2553 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749966 2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749969 2553 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749972 2553 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749975 2553 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749979 2553 flags.go:64] FLAG: --feature-gates="" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749983 2553 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749986 2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749989 2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749993 2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749996 2553 flags.go:64] FLAG: --healthz-port="10248" Jul 22 20:01:30.756305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.749999 2553 flags.go:64] FLAG: --help="false" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750002 2553 flags.go:64] FLAG: --hostname-override="ip-10-0-139-68.ec2.internal" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750005 2553 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750008 2553 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750012 2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750015 2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750019 2553 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750022 2553 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750025 2553 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750028 2553 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750031 2553 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750034 2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750037 2553 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750040 2553 flags.go:64] FLAG: --kube-reserved="" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750043 2553 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750046 2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750049 2553 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750052 2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750055 2553 flags.go:64] FLAG: --lock-file="" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750058 2553 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750060 2553 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750064 2553 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750069 2553 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 20:01:30.756983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750073 2553 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750076 2553 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750078 2553 flags.go:64] FLAG: --logging-format="text" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750081 2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750085 2553 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750088 2553 flags.go:64] FLAG: --manifest-url="" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750091 2553 flags.go:64] FLAG: --manifest-url-header="" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750096 2553 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750099 2553 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750103 2553 flags.go:64] FLAG: --max-pods="110" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750106 2553 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750110 2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750113 2553 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750116 2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750120 2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750122 2553 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750125 2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750134 2553 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750137 2553 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750140 2553 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750143 2553 flags.go:64] FLAG: --pod-cidr="" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750146 2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750152 2553 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750155 2553 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 20:01:30.757593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750161 2553 flags.go:64] FLAG: --pods-per-core="0" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750164 2553 flags.go:64] FLAG: --port="10250" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750167 2553 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750170 2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c3e0ef1bfe57017d" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750173 2553 flags.go:64] FLAG: --qos-reserved="" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750176 2553 flags.go:64] FLAG: --read-only-port="10255" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750179 2553 flags.go:64] FLAG: --register-node="true" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750182 2553 flags.go:64] FLAG: --register-with-taints="" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750186 2553 flags.go:64] FLAG: --registry-burst="10" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750190 2553 flags.go:64] FLAG: --registry-qps="5" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750193 2553 flags.go:64] FLAG: --reserved-cpus="" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750196 2553 flags.go:64] FLAG: --reserved-memory="" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750200 2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750203 2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750206 2553 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750209 2553 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750213 2553 flags.go:64] FLAG: --runonce="false" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750216 2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750219 2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750222 2553 flags.go:64] FLAG: --seccomp-default="false" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750225 2553 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750228 2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750231 2553 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750234 2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750237 2553 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750240 2553 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 20:01:30.758276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750244 2553 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750247 2553 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750250 2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750253 2553 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750257 2553 flags.go:64] FLAG: --system-cgroups="" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750259 2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750267 2553 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750270 2553 flags.go:64] FLAG: --tls-cert-file="" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750274 2553 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750278 2553 flags.go:64] FLAG: --tls-min-version="" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750281 2553 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750284 2553 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750287 2553 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750290 2553 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750293 2553 flags.go:64] FLAG: --v="2" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750298 2553 flags.go:64] FLAG: --version="false" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750304 2553 flags.go:64] FLAG: --vmodule="" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750308 2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.750311 2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750469 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750475 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750478 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750481 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 20:01:30.758942 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750484 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750487 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750490 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750493 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750496 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750500 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750503 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750506 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750510 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750514 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750517 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750520 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750523 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750526 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750529 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750536 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750539 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750543 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750546 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 20:01:30.759517 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750549 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750552 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750555 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750557 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750560 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750563 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750566 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750569 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750572 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750575 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750577 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750580 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750582 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750585 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750588 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750591 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750595 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750598 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750601 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750605 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 20:01:30.759996 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750607 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750611 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750613 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750617 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750620 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750622 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750625 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750628 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750634 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750637 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750640 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750643 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750650 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750653 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750657 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750660 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750662 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750665 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750669 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750672 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 20:01:30.760566 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750676 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750678 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750681 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750684 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750686 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750689 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750692 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750694 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750697 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750699 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750702 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750705 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750707 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750710 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750712 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750715 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750718 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750721 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750723 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 20:01:30.761060 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750727 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750734 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750737 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750739 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750742 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750747 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750750 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750752 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750755 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750758 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750760 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750763 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750765 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750768 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750772 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750774 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 20:01:30.761547 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.750777 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.751432 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.759341 2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.759367 2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.760124 2553 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.760135 2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760205 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760212 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760215 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760218 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760222 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760225 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760228 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760230 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760233 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 20:01:30.761949 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760236 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760238 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760241 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760245 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760247 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760251 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760253 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760255 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760258 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760261 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760264 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760266 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760269 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760272 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760274 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760278 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760282 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760284 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760287 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760290 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 20:01:30.762331 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760294 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760297 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760300 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760303 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760306 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760309 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760312 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760315 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760317 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760320 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760323 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760326 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760329 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760331 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760333 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760337 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760340 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760343 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760345 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760348 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 20:01:30.762841 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760351 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760353 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760356 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760358 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760361 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760363 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760366 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760369 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760373 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760377 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760381 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760384 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760388 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760391 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760394 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760397 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760400 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760403 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760422 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 20:01:30.763333 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760426 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760429 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760431 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760434 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760437 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760440 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760443 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760445 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760448 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760451 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760453 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760457 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760459 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760462 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760464 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760467 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760470 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760473 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760475 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760478 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 20:01:30.763875 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760481 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760483 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760487 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760491 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760496 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760502 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760506 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760509 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760512 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760515 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760518 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.760524 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760653 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760658 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760661 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 20:01:30.764358 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760665 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760668 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760671 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760673 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760676 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760679 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760682 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760684 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760687 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760689 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760692 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760694 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760697 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760700 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760702 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760705 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760708 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760710 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760713 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 20:01:30.764761 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760716 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760719 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760721 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760725 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760727 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760730 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760734 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760739 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760742 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760745 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760748 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760751 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760755 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760757 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760760 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760763 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760766 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760770 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760773 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 20:01:30.765231 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760775 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760778 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760781 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760784 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760787 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760789 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760792 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760795 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760798 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760800 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760803 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760805 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760808 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760810 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760813 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760816 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760819 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760823 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760826 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 20:01:30.765810 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760829 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760834 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760837 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760841 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760844 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760847 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760849 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760852 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760855 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760857 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760860 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760862 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760865 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760867 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760870 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760872 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760875 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760877 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760880 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760882 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 20:01:30.766275 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760885 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760888 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760890 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760893 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760896 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760899 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760902 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760904 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760907 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760911 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760913 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760916 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760918 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760921 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760923 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760926 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760929 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760931 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 22 20:01:30.766779 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:30.760934 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 20:01:30.767215 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.760939 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 20:01:30.767215 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.761097 2553 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 20:01:30.767215 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.763389 2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 20:01:30.767215 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.764756 2553 server.go:1019] "Starting client certificate rotation" Jul 22 20:01:30.767215 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.764873 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 20:01:30.767215 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.764923 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 20:01:30.768078 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.768039 2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 20:01:30.769831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.769800 2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 20:01:30.782098 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.782072 2553 log.go:25] "Validated CRI v1 runtime API" Jul 22 20:01:30.786421 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.786382 2553 log.go:25] "Validated CRI v1 image API" Jul 22 20:01:30.787493 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.787462 2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 20:01:30.792855 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.792820 2553 fs.go:135] Filesystem UUIDs: map[4ee8795f-d266-42c3-80d6-1f4cb2879a68:/dev/nvme0n1p3 595d134a-506f-4d4b-95f7-e531914609ab:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 22 20:01:30.792855 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.792849 2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 20:01:30.793816 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.793795 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 20:01:30.799436 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.799265 2553 manager.go:216] Machine: {Timestamp:2026-07-22 20:01:30.796829939 +0000 UTC m=+0.353573625 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090286 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e0587cd2eb54d8fa9638ebc287417 SystemUUID:ec2e0587-cd2e-b54d-8fa9-638ebc287417 BootID:e811b460-baa9-44dc-95f2-da32ac1f6445 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0e:69:47:b5:45 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0e:69:47:b5:45 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:83:77:1c:2a:94 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 20:01:30.799436 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.799404 2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 20:01:30.799567 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.799544 2553 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 20:01:30.800634 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.800594 2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 20:01:30.800805 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.800641 2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-68.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 20:01:30.801396 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.801381 2553 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 20:01:30.801451 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.801397 2553 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 20:01:30.801451 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.801406 2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 20:01:30.802217 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.802199 2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 20:01:30.803464 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.803447 2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 20:01:30.806822 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.806805 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 22 20:01:30.806972 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.806961 2553 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 20:01:30.807014 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.806997 2553 kubelet.go:486] "Attempting to sync node with API server" Jul 22 20:01:30.807014 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.807007 2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 20:01:30.807069 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.807021 2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 20:01:30.807069 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.807032 2553 kubelet.go:398] "Adding apiserver pod source" Jul 22 20:01:30.807069 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.807042 2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 20:01:30.807522 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.807508 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 20:01:30.807587 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.807531 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 20:01:30.810828 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.810787 2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 20:01:30.812442 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.812405 2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 20:01:30.812548 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.812458 2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 20:01:30.813089 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813075 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 20:01:30.813154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813093 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 20:01:30.813154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813101 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 20:01:30.813154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813110 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 20:01:30.813154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813120 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 20:01:30.813154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813125 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 20:01:30.813294 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813232 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 20:01:30.813294 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813239 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 20:01:30.813294 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813247 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 20:01:30.813294 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813252 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 20:01:30.813294 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813268 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 20:01:30.813294 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813275 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 20:01:30.813930 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813919 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 20:01:30.813982 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.813932 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 20:01:30.817760 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.817740 2553 server.go:1268] "Started kubelet" Jul 22 20:01:30.817881 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.817836 2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 20:01:30.818179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.817848 2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 20:01:30.818179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.818070 2553 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 20:01:30.818735 ip-10-0-139-68 systemd[1]: Started Kubernetes Kubelet. Jul 22 20:01:30.819304 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.819097 2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-68.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 20:01:30.819304 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.819201 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 20:01:30.819466 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.819450 2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 20:01:30.819644 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.819613 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 20:01:30.819907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.819881 2553 server.go:310] "Adding debug handlers to kubelet server" Jul 22 20:01:30.823384 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.823345 2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 20:01:30.826514 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.825564 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-68.ec2.internal.18c4b51d89e4aee6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-68.ec2.internal,UID:ip-10-0-139-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-68.ec2.internal,},FirstTimestamp:2026-07-22 20:01:30.817703654 +0000 UTC m=+0.374447340,LastTimestamp:2026-07-22 20:01:30.817703654 +0000 UTC m=+0.374447340,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-68.ec2.internal,}" Jul 22 20:01:30.827785 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.827758 2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 20:01:30.827907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.827764 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 20:01:30.827907 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.827900 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:30.828433 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.828395 2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 20:01:30.828433 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.828396 2553 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 20:01:30.828574 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.828439 2553 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 20:01:30.828574 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.828521 2553 reconstruct.go:97] "Volume reconstruction finished" Jul 22 20:01:30.828574 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.828531 2553 reconciler.go:29] "Reconciler: start to sync state" Jul 22 20:01:30.830587 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.830547 2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 20:01:30.830587 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.830586 2553 factory.go:55] Registering systemd factory Jul 22 20:01:30.830587 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.830595 2553 factory.go:223] Registration of the systemd container factory successfully Jul 22 20:01:30.831081 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.831063 2553 factory.go:153] Registering CRI-O factory Jul 22 20:01:30.831206 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.831195 2553 factory.go:223] Registration of the crio container factory successfully Jul 22 20:01:30.831294 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.831287 2553 factory.go:103] Registering Raw factory Jul 22 20:01:30.831361 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.831354 2553 manager.go:1196] Started watching for new ooms in manager Jul 22 20:01:30.832262 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.832246 2553 manager.go:349] Starting recovery of all containers Jul 22 20:01:30.833758 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.833723 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 20:01:30.836428 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.836352 2553 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 20:01:30.840871 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.840842 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fcm76" Jul 22 20:01:30.842904 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.842881 2553 manager.go:354] Recovery completed Jul 22 20:01:30.844595 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.844572 2553 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 22 20:01:30.847917 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.847899 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 20:01:30.848648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.848632 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fcm76" Jul 22 20:01:30.851602 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.851577 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 22 20:01:30.851685 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.851611 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 20:01:30.851685 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.851625 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 22 20:01:30.852207 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.852188 2553 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 20:01:30.852207 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.852206 2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 20:01:30.852308 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.852207 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 20:01:30.852308 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.852229 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 22 20:01:30.853014 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.852996 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 22 20:01:30.853089 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.853024 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 20:01:30.853089 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.853036 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 22 20:01:30.853510 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.853493 2553 policy_none.go:49] "None policy: Start" Jul 22 20:01:30.853606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.853518 2553 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 20:01:30.853606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.853529 2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 20:01:30.853606 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.853482 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-68.ec2.internal.18c4b51d8be9ce5c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-68.ec2.internal,UID:ip-10-0-139-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-68.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-68.ec2.internal,},FirstTimestamp:2026-07-22 20:01:30.85159382 +0000 UTC m=+0.408337506,LastTimestamp:2026-07-22 20:01:30.85159382 +0000 UTC m=+0.408337506,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-68.ec2.internal,}" Jul 22 20:01:30.854668 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.854655 2553 policy_none.go:47] "Start" Jul 22 20:01:30.894392 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.894185 2553 manager.go:337] "Starting Device Plugin manager" Jul 22 20:01:30.903582 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.894427 2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 20:01:30.903582 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.894439 2553 server.go:87] "Starting device plugin registration server" Jul 22 20:01:30.903582 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.894702 2553 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 20:01:30.903582 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.894712 2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 20:01:30.903582 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.894885 2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 20:01:30.903582 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.894980 2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 20:01:30.903582 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.894989 2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 20:01:30.903582 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.895529 2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 20:01:30.903582 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:30.895587 2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:30.995513 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.995405 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 20:01:30.996619 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.996600 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 22 20:01:30.996739 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.996635 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 20:01:30.996739 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.996648 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 22 20:01:30.996739 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.996675 2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-68.ec2.internal" Jul 22 20:01:30.998797 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:30.998767 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 20:01:31.000033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.000008 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 20:01:31.000157 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.000049 2553 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 20:01:31.000157 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.000076 2553 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 20:01:31.000157 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.000116 2553 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 20:01:31.002369 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.002340 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 20:01:31.006618 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.006589 2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.006618 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.006624 2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-68.ec2.internal\": node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.049585 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.048229 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.100332 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.100235 2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal"] Jul 22 20:01:31.100526 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.100378 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 20:01:31.101386 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.101369 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 22 20:01:31.101478 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.101404 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 20:01:31.101478 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.101436 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 22 20:01:31.102966 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.102950 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 20:01:31.103097 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.103077 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.103136 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.103121 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 20:01:31.103878 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.103857 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 22 20:01:31.104000 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.103885 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 20:01:31.104000 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.103896 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 22 20:01:31.104000 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.103924 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 22 20:01:31.104000 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.103950 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 20:01:31.104000 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.103965 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 22 20:01:31.105168 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.105149 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.105232 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.105189 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 20:01:31.106217 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.106198 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 22 20:01:31.106320 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.106231 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 20:01:31.106320 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.106242 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 22 20:01:31.130724 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.130690 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1aa63fbc826c704af774266db16e0b0d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-68.ec2.internal\" (UID: \"1aa63fbc826c704af774266db16e0b0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.130724 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.130723 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/593fb3001f52cd17064333937c0b946a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"593fb3001f52cd17064333937c0b946a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.130941 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.130748 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/593fb3001f52cd17064333937c0b946a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"593fb3001f52cd17064333937c0b946a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.132502 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.132483 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-68.ec2.internal\" not found" node="ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.137749 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.137729 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-68.ec2.internal\" not found" node="ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.149316 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.149279 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.231995 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.231955 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/593fb3001f52cd17064333937c0b946a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"593fb3001f52cd17064333937c0b946a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.231995 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.231999 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1aa63fbc826c704af774266db16e0b0d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-68.ec2.internal\" (UID: \"1aa63fbc826c704af774266db16e0b0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.232230 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.232027 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/593fb3001f52cd17064333937c0b946a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"593fb3001f52cd17064333937c0b946a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.232230 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.232073 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/593fb3001f52cd17064333937c0b946a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"593fb3001f52cd17064333937c0b946a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.232230 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.232088 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1aa63fbc826c704af774266db16e0b0d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-68.ec2.internal\" (UID: \"1aa63fbc826c704af774266db16e0b0d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.232230 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.232073 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/593fb3001f52cd17064333937c0b946a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"593fb3001f52cd17064333937c0b946a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.250264 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.250188 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.351208 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.351152 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.434820 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.434778 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.442371 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.442334 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 22 20:01:31.451584 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.451550 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.552056 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.551946 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.652889 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.652837 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.753527 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.753478 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.764460 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.764406 2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 20:01:31.764632 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.764613 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 20:01:31.853766 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.853651 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 19:56:31 +0000 UTC" deadline="2028-01-02 01:03:52.43768458 +0000 UTC" Jul 22 20:01:31.853766 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.853699 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12677h2m20.583989686s" Jul 22 20:01:31.853766 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.853676 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.954568 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:31.954535 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:31.974241 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:31.974174 2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 20:01:32.055126 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:32.055076 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:32.069989 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:32.069942 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593fb3001f52cd17064333937c0b946a.slice/crio-fc65313821939dcf450dea69f400b92ee684f6721011812a4ebb7fcb50d6d645 WatchSource:0}: Error finding container fc65313821939dcf450dea69f400b92ee684f6721011812a4ebb7fcb50d6d645: Status 404 returned error can't find the container with id fc65313821939dcf450dea69f400b92ee684f6721011812a4ebb7fcb50d6d645 Jul 22 20:01:32.070387 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:32.070358 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa63fbc826c704af774266db16e0b0d.slice/crio-5449daff3e95b5be168e9dbc4b825a9315020251205796b8956f0cae4bc7d614 WatchSource:0}: Error finding container 5449daff3e95b5be168e9dbc4b825a9315020251205796b8956f0cae4bc7d614: Status 404 returned error can't find the container with id 5449daff3e95b5be168e9dbc4b825a9315020251205796b8956f0cae4bc7d614 Jul 22 20:01:32.075973 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.075949 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 20:01:32.155279 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:32.155183 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:32.256087 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:32.256047 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 22 20:01:32.313626 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.313595 2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 20:01:32.339208 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.339179 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 20:01:32.428912 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.428817 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 22 20:01:32.442446 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.442399 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 20:01:32.443583 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.443561 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 22 20:01:32.452440 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.452401 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 20:01:32.636923 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.636881 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 20:01:32.810481 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.810370 2553 apiserver.go:52] "Watching apiserver" Jul 22 20:01:32.817280 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.817249 2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 20:01:32.817814 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.817784 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-zq625","openshift-multus/multus-additional-cni-plugins-ss5ld","openshift-multus/network-metrics-daemon-4cnsx","openshift-network-diagnostics/network-check-target-5dll9","openshift-network-operator/iptables-alerter-rln9j","openshift-ovn-kubernetes/ovnkube-node-xpshq","kube-system/global-pull-secret-syncer-mxfbh","kube-system/konnectivity-agent-cmnc4","kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm","openshift-cluster-node-tuning-operator/tuned-7vrgs","openshift-image-registry/node-ca-rbd5p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal","openshift-multus/multus-msxvm"] Jul 22 20:01:32.819822 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.819795 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.821877 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.821849 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.823128 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.822816 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 20:01:32.823128 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.822834 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 20:01:32.823128 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.822935 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9nptm\"" Jul 22 20:01:32.823128 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.822816 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 20:01:32.823128 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.822822 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 20:01:32.823128 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.823085 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 20:01:32.823583 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.823176 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 20:01:32.823965 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.823923 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 20:01:32.824083 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.824012 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 20:01:32.824372 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.824347 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.826263 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.826229 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 20:01:32.826439 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.826229 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p2jcx\"" Jul 22 20:01:32.826439 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.826388 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 20:01:32.826567 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.826530 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:32.829040 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.828382 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 20:01:32.829040 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.828434 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 20:01:32.829040 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.828587 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 20:01:32.829040 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.828790 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-49gqp\"" Jul 22 20:01:32.829040 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.828440 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 20:01:32.829388 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.829275 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 20:01:32.829693 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.829654 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.830164 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.830110 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mzflp\"" Jul 22 20:01:32.831522 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.831469 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 20:01:32.832106 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.832081 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rcp6c\"" Jul 22 20:01:32.832221 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.832129 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 20:01:32.833001 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.832911 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 20:01:32.833001 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.832931 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 20:01:32.833618 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.833481 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:32.833618 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:32.833601 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:32.835140 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.835088 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:32.835279 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:32.835230 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:32.836557 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.836534 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:32.838887 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.838472 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:32.839171 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.839124 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 20:01:32.839171 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.839144 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bd8jz\"" Jul 22 20:01:32.839171 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.839160 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 20:01:32.839384 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.839184 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 20:01:32.840167 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.840144 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.840815 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.840779 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 20:01:32.841525 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.841319 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 20:01:32.841525 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.841323 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kplbk\"" Jul 22 20:01:32.841525 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.841371 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 20:01:32.841853 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.841832 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:32.841972 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:32.841937 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:32.842727 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.842704 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-646c2\"" Jul 22 20:01:32.842877 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.842769 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 20:01:32.842877 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.842797 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 20:01:32.843034 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.842948 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-system-cni-dir\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.843034 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.842989 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-var-lib-cni-bin\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.843034 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843023 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-conf-dir\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.843191 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843085 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-daemon-config\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.843191 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843121 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad9034e0-a2b0-4bb7-bef5-b124b545dc59-serviceca\") pod \"node-ca-rbd5p\" (UID: \"ad9034e0-a2b0-4bb7-bef5-b124b545dc59\") " pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.843191 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843152 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-var-lib-openvswitch\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.843357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843190 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-run-k8s-cni-cncf-io\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.843357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843217 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3eacd02a-086d-4da8-8ccf-d05fc6ac4fac-host-slash\") pod \"iptables-alerter-rln9j\" (UID: \"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac\") " pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.843357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843241 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-systemd-units\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.843357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843281 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-run-systemd\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.843357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843310 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-node-log\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.843357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843339 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-cni-netd\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.843643 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843365 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.843643 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843395 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-etc-kubernetes\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.843643 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843471 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvbwc\" (UniqueName: \"kubernetes.io/projected/ad9034e0-a2b0-4bb7-bef5-b124b545dc59-kube-api-access-jvbwc\") pod \"node-ca-rbd5p\" (UID: \"ad9034e0-a2b0-4bb7-bef5-b124b545dc59\") " pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.843643 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843495 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:32.843643 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843501 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdvqw\" (UniqueName: \"kubernetes.io/projected/4da17f54-06b4-4750-9f1b-3304f6686447-kube-api-access-pdvqw\") pod \"node-resolver-zq625\" (UID: \"4da17f54-06b4-4750-9f1b-3304f6686447\") " pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843642 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-kubelet\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843669 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-run-openvswitch\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843695 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-ovnkube-script-lib\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843719 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-cni-dir\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843744 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-var-lib-cni-multus\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843768 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-run-netns\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843804 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-cni-bin\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843837 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-ovnkube-config\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843890 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-env-overrides\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843931 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-ovn-node-metrics-cert\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.843964 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3eacd02a-086d-4da8-8ccf-d05fc6ac4fac-iptables-alerter-script\") pod \"iptables-alerter-rln9j\" (UID: \"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac\") " pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.844033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844015 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-socket-dir-parent\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844081 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-var-lib-kubelet\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844111 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-hostroot\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844160 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-run-multus-certs\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844181 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr5mn\" (UniqueName: \"kubernetes.io/projected/e0704b41-003f-4d1f-9aed-c610af2ddaae-kube-api-access-hr5mn\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844203 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmvtw\" (UniqueName: \"kubernetes.io/projected/3eacd02a-086d-4da8-8ccf-d05fc6ac4fac-kube-api-access-dmvtw\") pod \"iptables-alerter-rln9j\" (UID: \"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac\") " pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844229 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-cnibin\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844254 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0704b41-003f-4d1f-9aed-c610af2ddaae-cni-binary-copy\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844290 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad9034e0-a2b0-4bb7-bef5-b124b545dc59-host\") pod \"node-ca-rbd5p\" (UID: \"ad9034e0-a2b0-4bb7-bef5-b124b545dc59\") " pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844321 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4da17f54-06b4-4750-9f1b-3304f6686447-hosts-file\") pod \"node-resolver-zq625\" (UID: \"4da17f54-06b4-4750-9f1b-3304f6686447\") " pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844341 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4da17f54-06b4-4750-9f1b-3304f6686447-tmp-dir\") pod \"node-resolver-zq625\" (UID: \"4da17f54-06b4-4750-9f1b-3304f6686447\") " pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844356 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-slash\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844371 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-etc-openvswitch\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844386 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-os-release\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844402 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-log-socket\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844418 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-run-ovn-kubernetes\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844439 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z28hf\" (UniqueName: \"kubernetes.io/projected/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-kube-api-access-z28hf\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.844687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844475 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-run-netns\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.845581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.844569 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-run-ovn\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.845581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.845516 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-svdmf\"" Jul 22 20:01:32.845992 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.845815 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 20:01:32.846115 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.845987 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 20:01:32.867357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.867280 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kbgzd" Jul 22 20:01:32.876078 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.876014 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kbgzd" Jul 22 20:01:32.930385 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.930296 2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 20:01:32.945263 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945225 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-sys-fs\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:32.945477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945275 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6fpj\" (UniqueName: \"kubernetes.io/projected/b5419f3c-d6a5-4530-872d-08505f1c1c74-kube-api-access-v6fpj\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:32.945477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945308 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-system-cni-dir\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:32.945477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945340 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-cnibin\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:32.945477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945368 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-run-netns\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.945477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945397 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-run-ovn\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945425 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-dbus\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:32.945477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945469 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-system-cni-dir\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945491 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-var-lib-cni-bin\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945485 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-run-netns\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945512 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-conf-dir\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945558 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-conf-dir\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945556 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-var-lib-openvswitch\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945562 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-var-lib-cni-bin\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945583 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-system-cni-dir\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945599 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-kubelet-config\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945558 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-run-ovn\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945607 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-var-lib-openvswitch\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945636 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-systemd-units\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945667 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-run-systemd\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945700 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-run-systemd\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945706 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-node-log\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945716 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-systemd-units\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945762 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.945831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945795 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945823 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945850 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945862 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbwc\" (UniqueName: \"kubernetes.io/projected/ad9034e0-a2b0-4bb7-bef5-b124b545dc59-kube-api-access-jvbwc\") pod \"node-ca-rbd5p\" (UID: \"ad9034e0-a2b0-4bb7-bef5-b124b545dc59\") " pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945888 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-run-openvswitch\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945827 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-node-log\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945944 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-etc-selinux\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.945953 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-run-openvswitch\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946058 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rq48\" (UniqueName: \"kubernetes.io/projected/1f0ede88-8585-41c8-b162-a161c7f1522a-kube-api-access-5rq48\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946118 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dq6f\" (UniqueName: \"kubernetes.io/projected/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-kube-api-access-4dq6f\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946146 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-cni-dir\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946164 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-var-lib-cni-multus\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946186 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-env-overrides\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946210 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-ovn-node-metrics-cert\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946233 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-sysctl-d\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946236 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-var-lib-cni-multus\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.946759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946256 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mn72\" (UniqueName: \"kubernetes.io/projected/64990ede-146b-469a-92ea-7096303d4f1c-kube-api-access-2mn72\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946281 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3eacd02a-086d-4da8-8ccf-d05fc6ac4fac-iptables-alerter-script\") pod \"iptables-alerter-rln9j\" (UID: \"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac\") " pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946310 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-socket-dir-parent\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946335 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-var-lib-kubelet\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946360 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr5mn\" (UniqueName: \"kubernetes.io/projected/e0704b41-003f-4d1f-9aed-c610af2ddaae-kube-api-access-hr5mn\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946384 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-socket-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946405 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-kubernetes\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946429 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-run\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946459 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-cni-dir\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946469 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-lib-modules\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946472 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-socket-dir-parent\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946504 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-cnibin\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946524 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-var-lib-kubelet\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946532 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4da17f54-06b4-4750-9f1b-3304f6686447-hosts-file\") pod \"node-resolver-zq625\" (UID: \"4da17f54-06b4-4750-9f1b-3304f6686447\") " pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946561 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-run-ovn-kubernetes\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946598 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-cnibin\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946600 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946689 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-sys\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.947591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946693 2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946646 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4da17f54-06b4-4750-9f1b-3304f6686447-hosts-file\") pod \"node-resolver-zq625\" (UID: \"4da17f54-06b4-4750-9f1b-3304f6686447\") " pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946639 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-run-ovn-kubernetes\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946736 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-cni-binary-copy\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946766 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946804 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z28hf\" (UniqueName: \"kubernetes.io/projected/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-kube-api-access-z28hf\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946833 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946852 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5f9c0b2-40ce-4b3c-92df-28329cda5d03-konnectivity-ca\") pod \"konnectivity-agent-cmnc4\" (UID: \"d5f9c0b2-40ce-4b3c-92df-28329cda5d03\") " pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946873 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-daemon-config\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946893 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad9034e0-a2b0-4bb7-bef5-b124b545dc59-serviceca\") pod \"node-ca-rbd5p\" (UID: \"ad9034e0-a2b0-4bb7-bef5-b124b545dc59\") " pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946905 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-env-overrides\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946918 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-sysconfig\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946935 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64990ede-146b-469a-92ea-7096303d4f1c-tmp\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.946967 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-os-release\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947007 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-run-k8s-cni-cncf-io\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947041 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3eacd02a-086d-4da8-8ccf-d05fc6ac4fac-host-slash\") pod \"iptables-alerter-rln9j\" (UID: \"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac\") " pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947053 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-run-k8s-cni-cncf-io\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.948419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947007 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3eacd02a-086d-4da8-8ccf-d05fc6ac4fac-iptables-alerter-script\") pod \"iptables-alerter-rln9j\" (UID: \"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac\") " pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947068 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-cni-netd\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947110 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-cni-netd\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947111 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-modprobe-d\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947124 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3eacd02a-086d-4da8-8ccf-d05fc6ac4fac-host-slash\") pod \"iptables-alerter-rln9j\" (UID: \"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac\") " pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947149 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-etc-kubernetes\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947202 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdvqw\" (UniqueName: \"kubernetes.io/projected/4da17f54-06b4-4750-9f1b-3304f6686447-kube-api-access-pdvqw\") pod \"node-resolver-zq625\" (UID: \"4da17f54-06b4-4750-9f1b-3304f6686447\") " pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947236 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-kubelet\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947262 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-ovnkube-script-lib\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947302 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-kubelet\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947302 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-etc-kubernetes\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947296 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947454 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-run-netns\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947436 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad9034e0-a2b0-4bb7-bef5-b124b545dc59-serviceca\") pod \"node-ca-rbd5p\" (UID: \"ad9034e0-a2b0-4bb7-bef5-b124b545dc59\") " pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947508 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-run-netns\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947537 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-cni-bin\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947545 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0704b41-003f-4d1f-9aed-c610af2ddaae-multus-daemon-config\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947570 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-ovnkube-config\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.949299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947586 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-cni-bin\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947599 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-systemd\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947625 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-var-lib-kubelet\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947642 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-host\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947658 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64990ede-146b-469a-92ea-7096303d4f1c-etc-tuned\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947689 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-hostroot\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947710 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-run-multus-certs\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947733 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmvtw\" (UniqueName: \"kubernetes.io/projected/3eacd02a-086d-4da8-8ccf-d05fc6ac4fac-kube-api-access-dmvtw\") pod \"iptables-alerter-rln9j\" (UID: \"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac\") " pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947778 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-slash\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947787 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-hostroot\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947820 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-host-run-multus-certs\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947848 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-host-slash\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947821 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947890 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0704b41-003f-4d1f-9aed-c610af2ddaae-cni-binary-copy\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947915 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad9034e0-a2b0-4bb7-bef5-b124b545dc59-host\") pod \"node-ca-rbd5p\" (UID: \"ad9034e0-a2b0-4bb7-bef5-b124b545dc59\") " pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947943 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-ovnkube-script-lib\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947966 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad9034e0-a2b0-4bb7-bef5-b124b545dc59-host\") pod \"node-ca-rbd5p\" (UID: \"ad9034e0-a2b0-4bb7-bef5-b124b545dc59\") " pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947974 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4da17f54-06b4-4750-9f1b-3304f6686447-tmp-dir\") pod \"node-resolver-zq625\" (UID: \"4da17f54-06b4-4750-9f1b-3304f6686447\") " pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:32.950213 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.947994 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-etc-openvswitch\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948013 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-log-socket\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948042 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5f9c0b2-40ce-4b3c-92df-28329cda5d03-agent-certs\") pod \"konnectivity-agent-cmnc4\" (UID: \"d5f9c0b2-40ce-4b3c-92df-28329cda5d03\") " pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948048 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-etc-openvswitch\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948069 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-sysctl-conf\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948080 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-ovnkube-config\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948094 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-os-release\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948139 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-log-socket\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948167 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-registration-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948206 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0704b41-003f-4d1f-9aed-c610af2ddaae-os-release\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948260 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-device-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948296 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4da17f54-06b4-4750-9f1b-3304f6686447-tmp-dir\") pod \"node-resolver-zq625\" (UID: \"4da17f54-06b4-4750-9f1b-3304f6686447\") " pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:32.951302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.948410 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0704b41-003f-4d1f-9aed-c610af2ddaae-cni-binary-copy\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.951900 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.951318 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-ovn-node-metrics-cert\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.960495 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.960447 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvbwc\" (UniqueName: \"kubernetes.io/projected/ad9034e0-a2b0-4bb7-bef5-b124b545dc59-kube-api-access-jvbwc\") pod \"node-ca-rbd5p\" (UID: \"ad9034e0-a2b0-4bb7-bef5-b124b545dc59\") " pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:32.960645 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.960530 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr5mn\" (UniqueName: \"kubernetes.io/projected/e0704b41-003f-4d1f-9aed-c610af2ddaae-kube-api-access-hr5mn\") pod \"multus-msxvm\" (UID: \"e0704b41-003f-4d1f-9aed-c610af2ddaae\") " pod="openshift-multus/multus-msxvm" Jul 22 20:01:32.960722 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.960704 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z28hf\" (UniqueName: \"kubernetes.io/projected/875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc-kube-api-access-z28hf\") pod \"ovnkube-node-xpshq\" (UID: \"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc\") " pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:32.960771 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.960749 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmvtw\" (UniqueName: \"kubernetes.io/projected/3eacd02a-086d-4da8-8ccf-d05fc6ac4fac-kube-api-access-dmvtw\") pod \"iptables-alerter-rln9j\" (UID: \"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac\") " pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:32.960811 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:32.960790 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdvqw\" (UniqueName: \"kubernetes.io/projected/4da17f54-06b4-4750-9f1b-3304f6686447-kube-api-access-pdvqw\") pod \"node-resolver-zq625\" (UID: \"4da17f54-06b4-4750-9f1b-3304f6686447\") " pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:33.006109 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.006030 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" event={"ID":"1aa63fbc826c704af774266db16e0b0d","Type":"ContainerStarted","Data":"5449daff3e95b5be168e9dbc4b825a9315020251205796b8956f0cae4bc7d614"} Jul 22 20:01:33.007617 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.007570 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" event={"ID":"593fb3001f52cd17064333937c0b946a","Type":"ContainerStarted","Data":"fc65313821939dcf450dea69f400b92ee684f6721011812a4ebb7fcb50d6d645"} Jul 22 20:01:33.049215 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049163 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.049398 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049222 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-sys\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.049398 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049341 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-cni-binary-copy\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.049398 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049373 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.049398 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049339 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.049661 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049406 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:33.049661 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049378 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-sys\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.049661 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049581 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5f9c0b2-40ce-4b3c-92df-28329cda5d03-konnectivity-ca\") pod \"konnectivity-agent-cmnc4\" (UID: \"d5f9c0b2-40ce-4b3c-92df-28329cda5d03\") " pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:33.049661 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049622 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-sysconfig\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.049661 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049654 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64990ede-146b-469a-92ea-7096303d4f1c-tmp\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.049911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049680 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-os-release\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.049911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049708 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-modprobe-d\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.049911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049738 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:33.049911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049766 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-systemd\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.049911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049789 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-var-lib-kubelet\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.049911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049817 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-host\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.049911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049841 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64990ede-146b-469a-92ea-7096303d4f1c-etc-tuned\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.049911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049871 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.049911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049907 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5f9c0b2-40ce-4b3c-92df-28329cda5d03-agent-certs\") pod \"konnectivity-agent-cmnc4\" (UID: \"d5f9c0b2-40ce-4b3c-92df-28329cda5d03\") " pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049936 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-sysctl-conf\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049963 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-registration-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.049991 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-device-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050014 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-sys-fs\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050039 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6fpj\" (UniqueName: \"kubernetes.io/projected/b5419f3c-d6a5-4530-872d-08505f1c1c74-kube-api-access-v6fpj\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050064 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-system-cni-dir\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050087 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-cnibin\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050117 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-dbus\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050143 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-cni-binary-copy\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050149 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-kubelet-config\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050192 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050193 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050210 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5f9c0b2-40ce-4b3c-92df-28329cda5d03-konnectivity-ca\") pod \"konnectivity-agent-cmnc4\" (UID: \"d5f9c0b2-40ce-4b3c-92df-28329cda5d03\") " pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050221 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050251 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-os-release\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050286 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-sysconfig\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.050430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050323 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-etc-selinux\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050357 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rq48\" (UniqueName: \"kubernetes.io/projected/1f0ede88-8585-41c8-b162-a161c7f1522a-kube-api-access-5rq48\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050385 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dq6f\" (UniqueName: \"kubernetes.io/projected/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-kube-api-access-4dq6f\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050398 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-sysctl-conf\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050423 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-sysctl-d\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050449 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mn72\" (UniqueName: \"kubernetes.io/projected/64990ede-146b-469a-92ea-7096303d4f1c-kube-api-access-2mn72\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050461 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-registration-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050510 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-socket-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050538 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-kubernetes\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050541 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-device-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050564 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-run\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050590 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-lib-modules\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050600 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-sys-fs\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050664 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-sysctl-d\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050676 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-cnibin\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050685 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-dbus\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050713 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.051218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050740 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-etc-selinux\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050563 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-kubelet-config\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050938 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-systemd\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.050949 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.050986 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-modprobe-d\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.051063 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-var-lib-kubelet\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.051082 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret podName:74d38ca2-5c25-4236-a7b5-335fc07b9ca2 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:33.551029359 +0000 UTC m=+3.107773049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret") pod "global-pull-secret-syncer-mxfbh" (UID: "74d38ca2-5c25-4236-a7b5-335fc07b9ca2") : object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.051092 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.051276 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.051305 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-host\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.051347 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:33.551328712 +0000 UTC m=+3.108072404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.051402 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-etc-kubernetes\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.051448 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-lib-modules\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.051537 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-system-cni-dir\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.051549 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1f0ede88-8585-41c8-b162-a161c7f1522a-socket-dir\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.052007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.051590 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64990ede-146b-469a-92ea-7096303d4f1c-run\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.053430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.053381 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64990ede-146b-469a-92ea-7096303d4f1c-tmp\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.053579 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.053533 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5f9c0b2-40ce-4b3c-92df-28329cda5d03-agent-certs\") pod \"konnectivity-agent-cmnc4\" (UID: \"d5f9c0b2-40ce-4b3c-92df-28329cda5d03\") " pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:33.053660 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.053586 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64990ede-146b-469a-92ea-7096303d4f1c-etc-tuned\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.059379 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.059332 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 20:01:33.059379 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.059375 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 20:01:33.059675 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.059397 2553 projected.go:196] Error preparing data for projected volume kube-api-access-fgggl for pod openshift-network-diagnostics/network-check-target-5dll9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:33.059675 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.059535 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl podName:d47f328b-a9d1-45b5-b78e-a2b31af68583 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:33.559508904 +0000 UTC m=+3.116252597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fgggl" (UniqueName: "kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl") pod "network-check-target-5dll9" (UID: "d47f328b-a9d1-45b5-b78e-a2b31af68583") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:33.064210 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.064110 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mn72\" (UniqueName: \"kubernetes.io/projected/64990ede-146b-469a-92ea-7096303d4f1c-kube-api-access-2mn72\") pod \"tuned-7vrgs\" (UID: \"64990ede-146b-469a-92ea-7096303d4f1c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.064210 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.064108 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rq48\" (UniqueName: \"kubernetes.io/projected/1f0ede88-8585-41c8-b162-a161c7f1522a-kube-api-access-5rq48\") pod \"aws-ebs-csi-driver-node-s4mbm\" (UID: \"1f0ede88-8585-41c8-b162-a161c7f1522a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.064640 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.064607 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dq6f\" (UniqueName: \"kubernetes.io/projected/e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f-kube-api-access-4dq6f\") pod \"multus-additional-cni-plugins-ss5ld\" (UID: \"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f\") " pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.064780 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.064667 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6fpj\" (UniqueName: \"kubernetes.io/projected/b5419f3c-d6a5-4530-872d-08505f1c1c74-kube-api-access-v6fpj\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:33.137681 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.137639 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:01:33.148730 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.148682 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-msxvm" Jul 22 20:01:33.151308 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:33.151259 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875cfb8a_9928_4c87_9dcc_c8e08f5bdfcc.slice/crio-85564cebc4260ac17a45ddba0f38fba535dba2d08d7ee3d6f3ae11411fb2d10e WatchSource:0}: Error finding container 85564cebc4260ac17a45ddba0f38fba535dba2d08d7ee3d6f3ae11411fb2d10e: Status 404 returned error can't find the container with id 85564cebc4260ac17a45ddba0f38fba535dba2d08d7ee3d6f3ae11411fb2d10e Jul 22 20:01:33.156625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.156593 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rbd5p" Jul 22 20:01:33.162767 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.162731 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zq625" Jul 22 20:01:33.163227 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:33.163193 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0704b41_003f_4d1f_9aed_c610af2ddaae.slice/crio-d3e9827be2370f0823f8b31738465ac72b901025b6b6868a25c0b717d82867bf WatchSource:0}: Error finding container d3e9827be2370f0823f8b31738465ac72b901025b6b6868a25c0b717d82867bf: Status 404 returned error can't find the container with id d3e9827be2370f0823f8b31738465ac72b901025b6b6868a25c0b717d82867bf Jul 22 20:01:33.170289 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:33.170224 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9034e0_a2b0_4bb7_bef5_b124b545dc59.slice/crio-cbbfd74275d8a053fee001564f859810bc3e0bfa31355fa27eb53c8082fac0f5 WatchSource:0}: Error finding container cbbfd74275d8a053fee001564f859810bc3e0bfa31355fa27eb53c8082fac0f5: Status 404 returned error can't find the container with id cbbfd74275d8a053fee001564f859810bc3e0bfa31355fa27eb53c8082fac0f5 Jul 22 20:01:33.172097 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.172053 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rln9j" Jul 22 20:01:33.175175 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:33.175035 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4da17f54_06b4_4750_9f1b_3304f6686447.slice/crio-4df27e0e7f3721ff0a4ca692741196aa901d7c0f76853daee69cae190a6b728b WatchSource:0}: Error finding container 4df27e0e7f3721ff0a4ca692741196aa901d7c0f76853daee69cae190a6b728b: Status 404 returned error can't find the container with id 4df27e0e7f3721ff0a4ca692741196aa901d7c0f76853daee69cae190a6b728b Jul 22 20:01:33.180660 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.180619 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:33.183556 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:33.183506 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eacd02a_086d_4da8_8ccf_d05fc6ac4fac.slice/crio-dbbebd6bfd0a8f9570df48c6ea5b0417072b0a8c7811e87a51d09d2f8f51d65f WatchSource:0}: Error finding container dbbebd6bfd0a8f9570df48c6ea5b0417072b0a8c7811e87a51d09d2f8f51d65f: Status 404 returned error can't find the container with id dbbebd6bfd0a8f9570df48c6ea5b0417072b0a8c7811e87a51d09d2f8f51d65f Jul 22 20:01:33.190327 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.190295 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" Jul 22 20:01:33.191067 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:33.191036 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f9c0b2_40ce_4b3c_92df_28329cda5d03.slice/crio-b38ffc4dafec2692d18cf1494a7c3deb1b69f5b2746dfb125d63d8e4e59716d3 WatchSource:0}: Error finding container b38ffc4dafec2692d18cf1494a7c3deb1b69f5b2746dfb125d63d8e4e59716d3: Status 404 returned error can't find the container with id b38ffc4dafec2692d18cf1494a7c3deb1b69f5b2746dfb125d63d8e4e59716d3 Jul 22 20:01:33.200251 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:33.200212 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0ede88_8585_41c8_b162_a161c7f1522a.slice/crio-7ffb68dba258ae6fa7902e715518fa75a4a6c8201b35e9331dcc50b418239d8c WatchSource:0}: Error finding container 7ffb68dba258ae6fa7902e715518fa75a4a6c8201b35e9331dcc50b418239d8c: Status 404 returned error can't find the container with id 7ffb68dba258ae6fa7902e715518fa75a4a6c8201b35e9331dcc50b418239d8c Jul 22 20:01:33.202375 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.202353 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" Jul 22 20:01:33.208340 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.208307 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" Jul 22 20:01:33.217954 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:01:33.217929 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode369ae6e_7aa0_4b1e_b1ad_ccac4c3d868f.slice/crio-f4e9a69b9f0f32ee06c5732e529118d607adf7416483417c2e50e90e86cfddff WatchSource:0}: Error finding container f4e9a69b9f0f32ee06c5732e529118d607adf7416483417c2e50e90e86cfddff: Status 404 returned error can't find the container with id f4e9a69b9f0f32ee06c5732e529118d607adf7416483417c2e50e90e86cfddff Jul 22 20:01:33.554865 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.554757 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:33.554865 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.554844 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:33.555126 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.554959 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:33.555126 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.555063 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret podName:74d38ca2-5c25-4236-a7b5-335fc07b9ca2 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:34.555039518 +0000 UTC m=+4.111783214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret") pod "global-pull-secret-syncer-mxfbh" (UID: "74d38ca2-5c25-4236-a7b5-335fc07b9ca2") : object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:33.555126 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.554971 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:33.555126 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.555127 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:34.555113578 +0000 UTC m=+4.111857257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:33.655909 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.655857 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:33.656123 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.656049 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 20:01:33.656123 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.656078 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 20:01:33.656123 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.656090 2553 projected.go:196] Error preparing data for projected volume kube-api-access-fgggl for pod openshift-network-diagnostics/network-check-target-5dll9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:33.656302 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:33.656151 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl podName:d47f328b-a9d1-45b5-b78e-a2b31af68583 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:34.656132322 +0000 UTC m=+4.212875998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fgggl" (UniqueName: "kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl") pod "network-check-target-5dll9" (UID: "d47f328b-a9d1-45b5-b78e-a2b31af68583") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:33.879539 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.879358 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 19:56:33 +0000 UTC" deadline="2028-02-07 14:18:13.911642599 +0000 UTC" Jul 22 20:01:33.879539 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:33.879444 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13554h16m40.032202625s" Jul 22 20:01:34.017186 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.016656 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" event={"ID":"1f0ede88-8585-41c8-b162-a161c7f1522a","Type":"ContainerStarted","Data":"7ffb68dba258ae6fa7902e715518fa75a4a6c8201b35e9331dcc50b418239d8c"} Jul 22 20:01:34.021406 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.021355 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cmnc4" event={"ID":"d5f9c0b2-40ce-4b3c-92df-28329cda5d03","Type":"ContainerStarted","Data":"b38ffc4dafec2692d18cf1494a7c3deb1b69f5b2746dfb125d63d8e4e59716d3"} Jul 22 20:01:34.024599 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.024527 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rln9j" event={"ID":"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac","Type":"ContainerStarted","Data":"dbbebd6bfd0a8f9570df48c6ea5b0417072b0a8c7811e87a51d09d2f8f51d65f"} Jul 22 20:01:34.030461 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.030356 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-msxvm" event={"ID":"e0704b41-003f-4d1f-9aed-c610af2ddaae","Type":"ContainerStarted","Data":"d3e9827be2370f0823f8b31738465ac72b901025b6b6868a25c0b717d82867bf"} Jul 22 20:01:34.035648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.035505 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" event={"ID":"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc","Type":"ContainerStarted","Data":"85564cebc4260ac17a45ddba0f38fba535dba2d08d7ee3d6f3ae11411fb2d10e"} Jul 22 20:01:34.038292 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.038095 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" event={"ID":"64990ede-146b-469a-92ea-7096303d4f1c","Type":"ContainerStarted","Data":"195c6866175f03c2b4b28af51fd65fd0cc588a2c6907e6d5325ee71b31ccd12e"} Jul 22 20:01:34.041106 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.041001 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zq625" event={"ID":"4da17f54-06b4-4750-9f1b-3304f6686447","Type":"ContainerStarted","Data":"4df27e0e7f3721ff0a4ca692741196aa901d7c0f76853daee69cae190a6b728b"} Jul 22 20:01:34.048371 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.048216 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rbd5p" event={"ID":"ad9034e0-a2b0-4bb7-bef5-b124b545dc59","Type":"ContainerStarted","Data":"cbbfd74275d8a053fee001564f859810bc3e0bfa31355fa27eb53c8082fac0f5"} Jul 22 20:01:34.053969 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.053853 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" event={"ID":"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f","Type":"ContainerStarted","Data":"f4e9a69b9f0f32ee06c5732e529118d607adf7416483417c2e50e90e86cfddff"} Jul 22 20:01:34.570044 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.569907 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:34.570044 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.570013 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:34.570312 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:34.570120 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:34.570312 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:34.570139 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:34.570312 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:34.570210 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret podName:74d38ca2-5c25-4236-a7b5-335fc07b9ca2 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:36.570185305 +0000 UTC m=+6.126928986 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret") pod "global-pull-secret-syncer-mxfbh" (UID: "74d38ca2-5c25-4236-a7b5-335fc07b9ca2") : object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:34.570312 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:34.570237 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:36.570224752 +0000 UTC m=+6.126968443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:34.671428 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.671381 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:34.671678 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:34.671654 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 20:01:34.671765 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:34.671687 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 20:01:34.671765 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:34.671704 2553 projected.go:196] Error preparing data for projected volume kube-api-access-fgggl for pod openshift-network-diagnostics/network-check-target-5dll9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:34.671872 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:34.671778 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl podName:d47f328b-a9d1-45b5-b78e-a2b31af68583 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:36.671756962 +0000 UTC m=+6.228500652 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fgggl" (UniqueName: "kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl") pod "network-check-target-5dll9" (UID: "d47f328b-a9d1-45b5-b78e-a2b31af68583") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:34.880604 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.880344 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 19:56:33 +0000 UTC" deadline="2028-02-28 15:40:40.363530105 +0000 UTC" Jul 22 20:01:34.880604 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:34.880423 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14059h39m5.483115387s" Jul 22 20:01:35.001220 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:35.001176 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:35.001413 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:35.001351 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:35.001871 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:35.001846 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:35.001985 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:35.001964 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:35.002046 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:35.002034 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:35.002121 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:35.002105 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:35.062505 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:35.062452 2553 generic.go:356] "Generic (PLEG): container finished" podID="593fb3001f52cd17064333937c0b946a" containerID="5a16f33ab30f20c931efc6b2e522f55dd85caf1cab7db9d6e70c1df47572b9b2" exitCode=0 Jul 22 20:01:35.062689 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:35.062531 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" event={"ID":"593fb3001f52cd17064333937c0b946a","Type":"ContainerDied","Data":"5a16f33ab30f20c931efc6b2e522f55dd85caf1cab7db9d6e70c1df47572b9b2"} Jul 22 20:01:36.079822 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:36.078961 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" event={"ID":"593fb3001f52cd17064333937c0b946a","Type":"ContainerStarted","Data":"08ccab75a1d0ae17ef92c7a51c8b36d29b29547f34acef14204312ce96118caa"} Jul 22 20:01:36.095614 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:36.095539 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" podStartSLOduration=3.095515069 podStartE2EDuration="3.095515069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:36.094257607 +0000 UTC m=+5.651001307" watchObservedRunningTime="2026-07-22 20:01:36.095515069 +0000 UTC m=+5.652258766" Jul 22 20:01:36.592172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:36.591840 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:36.592172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:36.591929 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:36.592172 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:36.592032 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:36.592172 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:36.592081 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:36.592172 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:36.592118 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret podName:74d38ca2-5c25-4236-a7b5-335fc07b9ca2 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:40.592095369 +0000 UTC m=+10.148839058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret") pod "global-pull-secret-syncer-mxfbh" (UID: "74d38ca2-5c25-4236-a7b5-335fc07b9ca2") : object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:36.592172 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:36.592146 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:40.592130607 +0000 UTC m=+10.148874283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:36.693339 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:36.693269 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:36.693604 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:36.693559 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 20:01:36.693604 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:36.693600 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 20:01:36.693760 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:36.693622 2553 projected.go:196] Error preparing data for projected volume kube-api-access-fgggl for pod openshift-network-diagnostics/network-check-target-5dll9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:36.693760 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:36.693711 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl podName:d47f328b-a9d1-45b5-b78e-a2b31af68583 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:40.693683774 +0000 UTC m=+10.250427473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fgggl" (UniqueName: "kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl") pod "network-check-target-5dll9" (UID: "d47f328b-a9d1-45b5-b78e-a2b31af68583") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:37.000618 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:37.000518 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:37.000818 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:37.000686 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:37.000818 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:37.000742 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:37.000957 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:37.000871 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:37.000957 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:37.000937 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:37.001069 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:37.001028 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:39.001853 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:39.001093 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:39.001853 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:39.001252 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:39.001853 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:39.001565 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:39.001853 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:39.001669 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:39.001853 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:39.001711 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:39.001853 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:39.001814 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:40.627513 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:40.627449 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:40.628117 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:40.627543 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:40.628117 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:40.627638 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:40.628117 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:40.627713 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:40.628117 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:40.627732 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret podName:74d38ca2-5c25-4236-a7b5-335fc07b9ca2 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:48.627704744 +0000 UTC m=+18.184448425 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret") pod "global-pull-secret-syncer-mxfbh" (UID: "74d38ca2-5c25-4236-a7b5-335fc07b9ca2") : object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:40.628117 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:40.627773 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:48.627754085 +0000 UTC m=+18.184497776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:40.728871 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:40.728805 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:40.729125 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:40.729049 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 20:01:40.729125 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:40.729084 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 20:01:40.729125 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:40.729099 2553 projected.go:196] Error preparing data for projected volume kube-api-access-fgggl for pod openshift-network-diagnostics/network-check-target-5dll9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:40.729286 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:40.729181 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl podName:d47f328b-a9d1-45b5-b78e-a2b31af68583 nodeName:}" failed. No retries permitted until 2026-07-22 20:01:48.729157925 +0000 UTC m=+18.285901615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fgggl" (UniqueName: "kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl") pod "network-check-target-5dll9" (UID: "d47f328b-a9d1-45b5-b78e-a2b31af68583") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:41.002474 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:41.001961 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:41.002474 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:41.002114 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:41.002474 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:41.002163 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:41.002474 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:41.002291 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:41.002474 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:41.002350 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:41.002474 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:41.002466 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:43.002846 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:43.001994 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:43.002846 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:43.002139 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:43.002846 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:43.002575 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:43.002846 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:43.002671 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:43.002846 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:43.002728 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:43.002846 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:43.002810 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:45.001053 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:45.001013 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:45.001571 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:45.001151 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:45.001571 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:45.001013 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:45.001571 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:45.001258 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:45.001571 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:45.001300 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:45.001571 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:45.001351 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:47.001002 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:47.000944 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:47.001551 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:47.000955 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:47.001551 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:47.001111 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:47.001551 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:47.000944 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:47.001551 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:47.001220 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:47.001551 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:47.001324 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:48.694159 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:48.694120 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:48.694718 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:48.694205 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:48.694718 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:48.694260 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:48.694718 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:48.694314 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:48.694718 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:48.694344 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:04.694325296 +0000 UTC m=+34.251068968 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:01:48.694718 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:48.694364 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret podName:74d38ca2-5c25-4236-a7b5-335fc07b9ca2 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:04.694354142 +0000 UTC m=+34.251097815 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret") pod "global-pull-secret-syncer-mxfbh" (UID: "74d38ca2-5c25-4236-a7b5-335fc07b9ca2") : object "kube-system"/"original-pull-secret" not registered Jul 22 20:01:48.795219 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:48.795182 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:48.795425 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:48.795347 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 20:01:48.795425 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:48.795365 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 20:01:48.795425 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:48.795375 2553 projected.go:196] Error preparing data for projected volume kube-api-access-fgggl for pod openshift-network-diagnostics/network-check-target-5dll9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:48.795579 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:48.795453 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl podName:d47f328b-a9d1-45b5-b78e-a2b31af68583 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:04.795429211 +0000 UTC m=+34.352172897 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fgggl" (UniqueName: "kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl") pod "network-check-target-5dll9" (UID: "d47f328b-a9d1-45b5-b78e-a2b31af68583") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:01:49.001335 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:49.001285 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:49.001335 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:49.001336 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:49.001646 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:49.001285 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:49.001646 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:49.001438 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:49.001646 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:49.001590 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:49.001801 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:49.001690 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:51.001776 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:51.001735 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:51.002292 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:51.001856 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:51.002292 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:51.001864 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:51.002292 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:51.001971 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:51.002292 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:51.002040 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:51.002292 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:51.002104 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:53.004508 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:53.004474 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:53.004975 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:53.004513 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:53.004975 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:53.004474 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:53.004975 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:53.004611 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:53.004975 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:53.004699 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:53.004975 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:53.004789 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:54.123100 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.122719 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" event={"ID":"1aa63fbc826c704af774266db16e0b0d","Type":"ContainerStarted","Data":"b0e385d41ea57745189c94da76fac6e540c77ceeebae968ba120e1b4530fc0d3"} Jul 22 20:01:54.124672 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.124646 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" event={"ID":"64990ede-146b-469a-92ea-7096303d4f1c","Type":"ContainerStarted","Data":"77b28123e9504f0a11250a15d89901d487617863535a7954a64d89e15e77f3b2"} Jul 22 20:01:54.126536 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.126457 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zq625" event={"ID":"4da17f54-06b4-4750-9f1b-3304f6686447","Type":"ContainerStarted","Data":"0cf936bc8806ccd346cfeeae50de090b4971bdb23d3a4ea377764f0d5cad2932"} Jul 22 20:01:54.128184 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.128160 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rbd5p" event={"ID":"ad9034e0-a2b0-4bb7-bef5-b124b545dc59","Type":"ContainerStarted","Data":"c89f65275b0aa8272f8a649403c7a656c007c4c16c76db8ac40dfd7547b257c1"} Jul 22 20:01:54.129880 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.129858 2553 generic.go:356] "Generic (PLEG): container finished" podID="e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f" containerID="683e5be299ffd43554eef477dd4d26dced829f9232c70675b92357b6427c8d79" exitCode=0 Jul 22 20:01:54.129983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.129943 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" event={"ID":"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f","Type":"ContainerDied","Data":"683e5be299ffd43554eef477dd4d26dced829f9232c70675b92357b6427c8d79"} Jul 22 20:01:54.131641 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.131621 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" event={"ID":"1f0ede88-8585-41c8-b162-a161c7f1522a","Type":"ContainerStarted","Data":"a3e032b19fb3e597243fd37e3e891fa2dec8668a64f4b0ec9adc78afd017159c"} Jul 22 20:01:54.133396 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.133375 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cmnc4" event={"ID":"d5f9c0b2-40ce-4b3c-92df-28329cda5d03","Type":"ContainerStarted","Data":"c13b6e553b1796401e12d5521fa3ffa61ad2ddb10324c37496d54d230994272f"} Jul 22 20:01:54.135937 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.135912 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-msxvm" event={"ID":"e0704b41-003f-4d1f-9aed-c610af2ddaae","Type":"ContainerStarted","Data":"9a102cff97f808a57b53ab048d21978327de2eb1c5339aec28a52dc9c5e94243"} Jul 22 20:01:54.137809 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.137759 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" podStartSLOduration=21.137741466 podStartE2EDuration="21.137741466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:54.136931229 +0000 UTC m=+23.693674948" watchObservedRunningTime="2026-07-22 20:01:54.137741466 +0000 UTC m=+23.694485163" Jul 22 20:01:54.138586 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.138560 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" event={"ID":"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc","Type":"ContainerStarted","Data":"11332dab4ae9c02d0c53eda22a3681a61ffb23fcbd7941a345982192eff68e89"} Jul 22 20:01:54.149831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.149772 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rbd5p" podStartSLOduration=2.516162382 podStartE2EDuration="23.149750557s" totalImagesPullingTime="20.633588175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:54.149559544 +0000 UTC m=+23.706303260" watchObservedRunningTime="2026-07-22 20:01:54.149750557 +0000 UTC m=+23.706494254" Jul 22 20:01:54.163270 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.163219 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cmnc4" podStartSLOduration=1.543867246 podStartE2EDuration="22.163203156s" totalImagesPullingTime="20.61933591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:54.16311908 +0000 UTC m=+23.719862776" watchObservedRunningTime="2026-07-22 20:01:54.163203156 +0000 UTC m=+23.719946850" Jul 22 20:01:54.181949 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.181904 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7vrgs" podStartSLOduration=1.568813523 podStartE2EDuration="22.181888523s" totalImagesPullingTime="20.613075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:54.181715786 +0000 UTC m=+23.738459495" watchObservedRunningTime="2026-07-22 20:01:54.181888523 +0000 UTC m=+23.738632219" Jul 22 20:01:54.203942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.203891 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-msxvm" podStartSLOduration=2.497051171 podStartE2EDuration="23.203875269s" totalImagesPullingTime="20.706824098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:54.203283842 +0000 UTC m=+23.760027538" watchObservedRunningTime="2026-07-22 20:01:54.203875269 +0000 UTC m=+23.760618964" Jul 22 20:01:54.251440 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:54.250572 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zq625" podStartSLOduration=2.604827435 podStartE2EDuration="23.250549715s" totalImagesPullingTime="20.64572228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:54.226917078 +0000 UTC m=+23.783660788" watchObservedRunningTime="2026-07-22 20:01:54.250549715 +0000 UTC m=+23.807293407" Jul 22 20:01:55.000692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.000648 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:55.000990 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.000648 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:55.000990 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:55.000780 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:55.000990 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.000661 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:55.000990 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:55.000867 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:55.000990 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:55.000916 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:55.142985 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.142899 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rln9j" event={"ID":"3eacd02a-086d-4da8-8ccf-d05fc6ac4fac","Type":"ContainerStarted","Data":"21cd19b9b59f0ca3552adbed94f7d7dbb189c0501abb8e74f605d81b6c9ea402"} Jul 22 20:01:55.146329 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.146271 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" event={"ID":"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc","Type":"ContainerStarted","Data":"337d2b20d1471d1720b286528bc3effb3a5f898ee74bb4491e99620e4407744e"} Jul 22 20:01:55.146507 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.146367 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" event={"ID":"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc","Type":"ContainerStarted","Data":"6f5fe2d177062a4f1bda41b98df39c9d83490c459b35b990f3d671f2d6c226fa"} Jul 22 20:01:55.146507 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.146436 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" event={"ID":"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc","Type":"ContainerStarted","Data":"101046ebd3dedfcb9d567cbeff4f91aa732726baa5d22a6dcc8676b42643715e"} Jul 22 20:01:55.146507 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.146468 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" event={"ID":"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc","Type":"ContainerStarted","Data":"318b3c785e8847b2b0a588d6aa89687f1e592107fcda98ab5a27ba47d5a0b784"} Jul 22 20:01:55.146507 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.146502 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" event={"ID":"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc","Type":"ContainerStarted","Data":"98c9e96b02041d67d922a4d1faa6add7c3aff47f1fb54bb5721e4bd10cf0b553"} Jul 22 20:01:55.156743 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.156693 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rln9j" podStartSLOduration=3.54177231 podStartE2EDuration="24.156678432s" totalImagesPullingTime="20.614906122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:55.156531898 +0000 UTC m=+24.713275618" watchObservedRunningTime="2026-07-22 20:01:55.156678432 +0000 UTC m=+24.713422128" Jul 22 20:01:55.912560 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:55.912536 2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 20:01:56.150330 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:56.150238 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" event={"ID":"1f0ede88-8585-41c8-b162-a161c7f1522a","Type":"ContainerStarted","Data":"7723570db99bca9245bd3ba0335ea434e9887925bb94f5c3f8491870d2fff6cd"} Jul 22 20:01:56.910206 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:56.910012 2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T20:01:55.912554231Z","UUID":"5f449a82-f0e2-47c3-aed0-10bfd18bb27e","Handler":null,"Name":"","Endpoint":""} Jul 22 20:01:56.913094 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:56.913059 2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 20:01:56.913255 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:56.913110 2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 20:01:57.000606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:57.000568 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:57.000606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:57.000613 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:57.000846 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:57.000568 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:57.000846 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:57.000702 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:57.000846 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:57.000770 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:57.000846 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:57.000835 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:57.097747 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:57.097709 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:57.098383 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:57.098357 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:57.157791 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:57.157749 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" event={"ID":"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc","Type":"ContainerStarted","Data":"b8b3fdd6e0d6c2c306500ea5af1163db6b6bcdacac5cdc0d66cafa11ee4e31da"} Jul 22 20:01:58.162921 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:58.162728 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" event={"ID":"1f0ede88-8585-41c8-b162-a161c7f1522a","Type":"ContainerStarted","Data":"8473dc8d866f4412cfe49a0a99fd9959a9103a16776ca29fd30042d199fd971b"} Jul 22 20:01:58.180103 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:58.180046 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s4mbm" podStartSLOduration=2.450688654 podStartE2EDuration="26.18002795s" totalImagesPullingTime="23.729339296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:58.179386275 +0000 UTC m=+27.736129991" watchObservedRunningTime="2026-07-22 20:01:58.18002795 +0000 UTC m=+27.736771644" Jul 22 20:01:58.557359 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:58.557322 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:58.557601 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:58.557536 2553 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 22 20:01:58.558043 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:58.558013 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cmnc4" Jul 22 20:01:59.001144 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:59.001116 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:01:59.001243 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:59.001154 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:01:59.001243 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:59.001116 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:01:59.001309 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:59.001255 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:01:59.001371 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:59.001338 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:01:59.001552 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:01:59.001468 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:01:59.170243 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:59.170207 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" event={"ID":"875cfb8a-9928-4c87-9dcc-c8e08f5bdfcc","Type":"ContainerStarted","Data":"5a4083786d3378eeb0c543e4e24212c9fc7a5247d29111c6460a2e02a11fd4c0"} Jul 22 20:01:59.200604 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:01:59.200534 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" podStartSLOduration=7.460678437 podStartE2EDuration="28.200512315s" totalImagesPullingTime="20.739833878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:01:59.198977757 +0000 UTC m=+28.755721489" watchObservedRunningTime="2026-07-22 20:01:59.200512315 +0000 UTC m=+28.757256011" Jul 22 20:02:00.174510 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:00.174296 2553 generic.go:356] "Generic (PLEG): container finished" podID="e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f" containerID="2fb809639cdc7a97c2292cfc4c9ea07fffce1492f2eff14b71e6e76b5728e8e2" exitCode=0 Jul 22 20:02:00.175001 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:00.174386 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" event={"ID":"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f","Type":"ContainerDied","Data":"2fb809639cdc7a97c2292cfc4c9ea07fffce1492f2eff14b71e6e76b5728e8e2"} Jul 22 20:02:00.175001 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:00.174942 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:02:00.175702 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:00.175021 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:02:00.175702 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:00.175071 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:02:00.191570 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:00.191542 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:02:00.191730 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:00.191646 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:02:01.001326 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:01.001292 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:02:01.001623 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:01.001400 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:01.001623 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:01.001400 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:02:01.001623 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:01.001500 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:02:01.001800 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:01.001647 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:02:01.001800 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:01.001739 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:02:01.268699 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:01.268600 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mxfbh"] Jul 22 20:02:01.269194 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:01.268788 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:02:01.269194 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:01.268938 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:02:01.272893 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:01.272849 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4cnsx"] Jul 22 20:02:01.273074 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:01.272992 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:02:01.273140 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:01.273122 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:02:01.273617 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:01.273590 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5dll9"] Jul 22 20:02:01.273714 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:01.273690 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:01.273800 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:01.273777 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:02:03.000635 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:03.000595 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:03.001046 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:03.000595 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:02:03.001046 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:03.000722 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:02:03.001046 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:03.000815 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:02:03.001046 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:03.000601 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:02:03.001046 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:03.000971 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:02:03.183882 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:03.183847 2553 generic.go:356] "Generic (PLEG): container finished" podID="e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f" containerID="ce875af4b3ba1792599e4f632068ae49797e6e48d58b561a058098ac53ce3e12" exitCode=0 Jul 22 20:02:03.184042 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:03.183919 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" event={"ID":"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f","Type":"ContainerDied","Data":"ce875af4b3ba1792599e4f632068ae49797e6e48d58b561a058098ac53ce3e12"} Jul 22 20:02:04.709500 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:04.709198 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:02:04.709500 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:04.709354 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 20:02:04.709976 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:04.709545 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret podName:74d38ca2-5c25-4236-a7b5-335fc07b9ca2 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:36.709527818 +0000 UTC m=+66.266271491 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret") pod "global-pull-secret-syncer-mxfbh" (UID: "74d38ca2-5c25-4236-a7b5-335fc07b9ca2") : object "kube-system"/"original-pull-secret" not registered Jul 22 20:02:04.709976 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:04.709475 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:02:04.709976 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:04.709567 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:02:04.709976 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:04.709643 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:36.709625814 +0000 UTC m=+66.266369504 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 20:02:04.810127 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:04.810085 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:04.810298 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:04.810249 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 20:02:04.810298 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:04.810277 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 20:02:04.810298 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:04.810289 2553 projected.go:196] Error preparing data for projected volume kube-api-access-fgggl for pod openshift-network-diagnostics/network-check-target-5dll9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:02:04.810433 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:04.810341 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl podName:d47f328b-a9d1-45b5-b78e-a2b31af68583 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:36.810327391 +0000 UTC m=+66.367071063 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fgggl" (UniqueName: "kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl") pod "network-check-target-5dll9" (UID: "d47f328b-a9d1-45b5-b78e-a2b31af68583") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 20:02:05.000645 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:05.000607 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:02:05.000645 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:05.000644 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:05.000890 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:05.000607 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:02:05.000890 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:05.000749 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:02:05.000890 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:05.000850 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mxfbh" podUID="74d38ca2-5c25-4236-a7b5-335fc07b9ca2" Jul 22 20:02:05.001023 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:05.000903 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5dll9" podUID="d47f328b-a9d1-45b5-b78e-a2b31af68583" Jul 22 20:02:05.191427 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:05.191377 2553 generic.go:356] "Generic (PLEG): container finished" podID="e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f" containerID="5dc50cb82ec8c94b9e832578538633f21a9dcbacba63b91abefb960cfff5697e" exitCode=0 Jul 22 20:02:05.191644 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:05.191466 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" event={"ID":"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f","Type":"ContainerDied","Data":"5dc50cb82ec8c94b9e832578538633f21a9dcbacba63b91abefb960cfff5697e"} Jul 22 20:02:06.680153 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.680068 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeReady" Jul 22 20:02:06.680663 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.680193 2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 20:02:06.718848 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.718806 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz"] Jul 22 20:02:06.721199 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.721169 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:06.722833 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.722796 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-579b98c4-8z9rg"] Jul 22 20:02:06.724606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.724586 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.726176 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.726148 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-n9xf4\"" Jul 22 20:02:06.726293 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.726186 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 20:02:06.726293 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.726151 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 20:02:06.726598 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.726581 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 20:02:06.726771 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.726754 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 20:02:06.726983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.726967 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 20:02:06.727039 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.726968 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gzjk5\"" Jul 22 20:02:06.731614 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.731592 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 20:02:06.740071 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.740031 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz"] Jul 22 20:02:06.741717 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.741676 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5g5zk"] Jul 22 20:02:06.744856 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.744810 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hqpcf"] Jul 22 20:02:06.745030 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.744932 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5g5zk" Jul 22 20:02:06.747320 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.747294 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 20:02:06.747320 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.747309 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 20:02:06.748088 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.747906 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9cx5s\"" Jul 22 20:02:06.748088 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.747995 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-579b98c4-8z9rg"] Jul 22 20:02:06.748263 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.748150 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.749717 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.749679 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 20:02:06.750276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.750120 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 20:02:06.750448 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.750388 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 20:02:06.750670 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.750653 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zbn42\"" Jul 22 20:02:06.751711 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.751685 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5g5zk"] Jul 22 20:02:06.752648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.752622 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hqpcf"] Jul 22 20:02:06.825313 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825276 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sv45\" (UniqueName: \"kubernetes.io/projected/c5545d45-8b89-4f48-813c-6c73a905c56a-kube-api-access-6sv45\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.825313 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825314 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.825546 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825349 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-trusted-ca\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.825546 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825366 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-bound-sa-token\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.825546 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825435 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0f4e343-fae7-43f2-9cd7-15ea347781ad-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:06.825546 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825457 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da2a3c08-e156-44fb-8571-5c3add922c89-ca-trust-extracted\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.825546 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825475 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klsfj\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-kube-api-access-klsfj\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.825546 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825497 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c525512-dbbd-4e74-8b1e-28270cfb6554-cert\") pod \"ingress-canary-5g5zk\" (UID: \"8c525512-dbbd-4e74-8b1e-28270cfb6554\") " pod="openshift-ingress-canary/ingress-canary-5g5zk" Jul 22 20:02:06.825546 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825531 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:06.825801 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825562 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5545d45-8b89-4f48-813c-6c73a905c56a-config-volume\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.825801 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825586 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnfqr\" (UniqueName: \"kubernetes.io/projected/8c525512-dbbd-4e74-8b1e-28270cfb6554-kube-api-access-fnfqr\") pod \"ingress-canary-5g5zk\" (UID: \"8c525512-dbbd-4e74-8b1e-28270cfb6554\") " pod="openshift-ingress-canary/ingress-canary-5g5zk" Jul 22 20:02:06.825801 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825600 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.825801 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825646 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5545d45-8b89-4f48-813c-6c73a905c56a-tmp-dir\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.825801 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825688 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-installation-pull-secrets\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.825801 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825716 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-image-registry-private-configuration\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.825801 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.825735 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-registry-certificates\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.927046 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927004 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5545d45-8b89-4f48-813c-6c73a905c56a-tmp-dir\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.927046 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927047 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-installation-pull-secrets\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.927200 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927073 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-image-registry-private-configuration\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.927200 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927089 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-registry-certificates\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.927334 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927307 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sv45\" (UniqueName: \"kubernetes.io/projected/c5545d45-8b89-4f48-813c-6c73a905c56a-kube-api-access-6sv45\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.927405 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927351 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.927405 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927398 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-trusted-ca\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.927539 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927452 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-bound-sa-token\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.927539 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927480 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5545d45-8b89-4f48-813c-6c73a905c56a-tmp-dir\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.927539 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927486 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0f4e343-fae7-43f2-9cd7-15ea347781ad-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:06.927539 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927537 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da2a3c08-e156-44fb-8571-5c3add922c89-ca-trust-extracted\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.927741 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927565 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klsfj\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-kube-api-access-klsfj\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.927741 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927608 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c525512-dbbd-4e74-8b1e-28270cfb6554-cert\") pod \"ingress-canary-5g5zk\" (UID: \"8c525512-dbbd-4e74-8b1e-28270cfb6554\") " pod="openshift-ingress-canary/ingress-canary-5g5zk" Jul 22 20:02:06.927741 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927675 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:06.927741 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927734 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5545d45-8b89-4f48-813c-6c73a905c56a-config-volume\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.927934 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.927757 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-registry-certificates\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.928007 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfqr\" (UniqueName: \"kubernetes.io/projected/8c525512-dbbd-4e74-8b1e-28270cfb6554-kube-api-access-fnfqr\") pod \"ingress-canary-5g5zk\" (UID: \"8c525512-dbbd-4e74-8b1e-28270cfb6554\") " pod="openshift-ingress-canary/ingress-canary-5g5zk" Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.928042 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:06.928052 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:06.928074 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579b98c4-8z9rg: secret "image-registry-tls" not found Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:06.928138 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls podName:da2a3c08-e156-44fb-8571-5c3add922c89 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:07.428118797 +0000 UTC m=+36.984862493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls") pod "image-registry-579b98c4-8z9rg" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89") : secret "image-registry-tls" not found Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:06.928143 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.928165 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da2a3c08-e156-44fb-8571-5c3add922c89-ca-trust-extracted\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:06.928196 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls podName:c5545d45-8b89-4f48-813c-6c73a905c56a nodeName:}" failed. No retries permitted until 2026-07-22 20:02:07.428182227 +0000 UTC m=+36.984925914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls") pod "dns-default-hqpcf" (UID: "c5545d45-8b89-4f48-813c-6c73a905c56a") : secret "dns-default-metrics-tls" not found Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.928220 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5545d45-8b89-4f48-813c-6c73a905c56a-config-volume\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:06.928283 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 20:02:06.928430 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:06.928316 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert podName:b0f4e343-fae7-43f2-9cd7-15ea347781ad nodeName:}" failed. No retries permitted until 2026-07-22 20:02:07.428307089 +0000 UTC m=+36.985050765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-r7mrz" (UID: "b0f4e343-fae7-43f2-9cd7-15ea347781ad") : secret "networking-console-plugin-cert" not found Jul 22 20:02:06.928933 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.928608 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-trusted-ca\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.929176 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.929145 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0f4e343-fae7-43f2-9cd7-15ea347781ad-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:06.932371 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.932298 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-installation-pull-secrets\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.932371 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.932298 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-image-registry-private-configuration\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.932371 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.932305 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c525512-dbbd-4e74-8b1e-28270cfb6554-cert\") pod \"ingress-canary-5g5zk\" (UID: \"8c525512-dbbd-4e74-8b1e-28270cfb6554\") " pod="openshift-ingress-canary/ingress-canary-5g5zk" Jul 22 20:02:06.936248 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.936209 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sv45\" (UniqueName: \"kubernetes.io/projected/c5545d45-8b89-4f48-813c-6c73a905c56a-kube-api-access-6sv45\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:06.936248 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.936244 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-bound-sa-token\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.936489 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.936465 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klsfj\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-kube-api-access-klsfj\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:06.936661 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:06.936647 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnfqr\" (UniqueName: \"kubernetes.io/projected/8c525512-dbbd-4e74-8b1e-28270cfb6554-kube-api-access-fnfqr\") pod \"ingress-canary-5g5zk\" (UID: \"8c525512-dbbd-4e74-8b1e-28270cfb6554\") " pod="openshift-ingress-canary/ingress-canary-5g5zk" Jul 22 20:02:07.000344 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.000306 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:02:07.000567 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.000306 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:02:07.000567 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.000306 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:07.002665 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.002634 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 20:02:07.002830 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.002797 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 20:02:07.003008 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.002989 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nrd5t\"" Jul 22 20:02:07.003073 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.003044 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l85mx\"" Jul 22 20:02:07.003157 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.003137 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 20:02:07.003220 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.003137 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 20:02:07.058262 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.058204 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5g5zk" Jul 22 20:02:07.200166 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.200067 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5g5zk"] Jul 22 20:02:07.204767 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:02:07.204732 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c525512_dbbd_4e74_8b1e_28270cfb6554.slice/crio-04c362a6db8c056943fcde78aaf1a285cdae7622f9867a80e29ee673fd77d58b WatchSource:0}: Error finding container 04c362a6db8c056943fcde78aaf1a285cdae7622f9867a80e29ee673fd77d58b: Status 404 returned error can't find the container with id 04c362a6db8c056943fcde78aaf1a285cdae7622f9867a80e29ee673fd77d58b Jul 22 20:02:07.433772 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.433726 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:07.433930 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.433804 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:07.433930 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:07.433847 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:07.433930 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:07.433885 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 20:02:07.433930 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:07.433907 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579b98c4-8z9rg: secret "image-registry-tls" not found Jul 22 20:02:07.433930 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:07.433925 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 20:02:07.434128 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:07.433891 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 20:02:07.434128 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:07.433957 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls podName:da2a3c08-e156-44fb-8571-5c3add922c89 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:08.433940524 +0000 UTC m=+37.990684196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls") pod "image-registry-579b98c4-8z9rg" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89") : secret "image-registry-tls" not found Jul 22 20:02:07.434128 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:07.433989 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls podName:c5545d45-8b89-4f48-813c-6c73a905c56a nodeName:}" failed. No retries permitted until 2026-07-22 20:02:08.433977962 +0000 UTC m=+37.990721636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls") pod "dns-default-hqpcf" (UID: "c5545d45-8b89-4f48-813c-6c73a905c56a") : secret "dns-default-metrics-tls" not found Jul 22 20:02:07.434128 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:07.434002 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert podName:b0f4e343-fae7-43f2-9cd7-15ea347781ad nodeName:}" failed. No retries permitted until 2026-07-22 20:02:08.433995754 +0000 UTC m=+37.990739426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-r7mrz" (UID: "b0f4e343-fae7-43f2-9cd7-15ea347781ad") : secret "networking-console-plugin-cert" not found Jul 22 20:02:08.200673 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:08.200616 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5g5zk" event={"ID":"8c525512-dbbd-4e74-8b1e-28270cfb6554","Type":"ContainerStarted","Data":"04c362a6db8c056943fcde78aaf1a285cdae7622f9867a80e29ee673fd77d58b"} Jul 22 20:02:08.443903 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:08.443863 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:08.444072 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:08.443916 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:08.444072 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:08.443987 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:08.444072 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:08.444029 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 20:02:08.444221 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:08.444073 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 20:02:08.444221 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:08.444088 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579b98c4-8z9rg: secret "image-registry-tls" not found Jul 22 20:02:08.444221 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:08.444098 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls podName:c5545d45-8b89-4f48-813c-6c73a905c56a nodeName:}" failed. No retries permitted until 2026-07-22 20:02:10.444077503 +0000 UTC m=+40.000821177 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls") pod "dns-default-hqpcf" (UID: "c5545d45-8b89-4f48-813c-6c73a905c56a") : secret "dns-default-metrics-tls" not found Jul 22 20:02:08.444221 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:08.444122 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls podName:da2a3c08-e156-44fb-8571-5c3add922c89 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:10.444111976 +0000 UTC m=+40.000855654 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls") pod "image-registry-579b98c4-8z9rg" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89") : secret "image-registry-tls" not found Jul 22 20:02:08.444221 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:08.444127 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 20:02:08.444221 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:08.444198 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert podName:b0f4e343-fae7-43f2-9cd7-15ea347781ad nodeName:}" failed. No retries permitted until 2026-07-22 20:02:10.444180619 +0000 UTC m=+40.000924304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-r7mrz" (UID: "b0f4e343-fae7-43f2-9cd7-15ea347781ad") : secret "networking-console-plugin-cert" not found Jul 22 20:02:10.462198 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:10.462136 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:10.462198 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:10.462203 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:10.462771 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:10.462288 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:10.462771 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:10.462341 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 20:02:10.462771 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:10.462447 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 20:02:10.462771 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:10.462490 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 20:02:10.462771 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:10.462508 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579b98c4-8z9rg: secret "image-registry-tls" not found Jul 22 20:02:10.462771 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:10.462449 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls podName:c5545d45-8b89-4f48-813c-6c73a905c56a nodeName:}" failed. No retries permitted until 2026-07-22 20:02:14.46242394 +0000 UTC m=+44.019167632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls") pod "dns-default-hqpcf" (UID: "c5545d45-8b89-4f48-813c-6c73a905c56a") : secret "dns-default-metrics-tls" not found Jul 22 20:02:10.462771 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:10.462561 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert podName:b0f4e343-fae7-43f2-9cd7-15ea347781ad nodeName:}" failed. No retries permitted until 2026-07-22 20:02:14.462547027 +0000 UTC m=+44.019290703 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-r7mrz" (UID: "b0f4e343-fae7-43f2-9cd7-15ea347781ad") : secret "networking-console-plugin-cert" not found Jul 22 20:02:10.462771 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:10.462579 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls podName:da2a3c08-e156-44fb-8571-5c3add922c89 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:14.462567246 +0000 UTC m=+44.019310919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls") pod "image-registry-579b98c4-8z9rg" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89") : secret "image-registry-tls" not found Jul 22 20:02:13.217089 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:13.217048 2553 generic.go:356] "Generic (PLEG): container finished" podID="e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f" containerID="625084d17b70f6cf269c3e15fddba88a93bc1df0100eca3da266f83a48f63cc9" exitCode=0 Jul 22 20:02:13.217585 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:13.217132 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" event={"ID":"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f","Type":"ContainerDied","Data":"625084d17b70f6cf269c3e15fddba88a93bc1df0100eca3da266f83a48f63cc9"} Jul 22 20:02:13.218692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:13.218670 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5g5zk" event={"ID":"8c525512-dbbd-4e74-8b1e-28270cfb6554","Type":"ContainerStarted","Data":"bfba61f22f5fd7889070eac5e551f74d96ce0815e6384dbe1f77b6124b6dcbaf"} Jul 22 20:02:13.256831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:13.256704 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5g5zk" podStartSLOduration=1.6122347970000002 podStartE2EDuration="7.256679503s" totalImagesPullingTime="5.644444706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:02:13.255050221 +0000 UTC m=+42.811793917" watchObservedRunningTime="2026-07-22 20:02:13.256679503 +0000 UTC m=+42.813423201" Jul 22 20:02:14.224299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:14.224263 2553 generic.go:356] "Generic (PLEG): container finished" podID="e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f" containerID="dee57af68bd1651fb325caede114e960cbfc4a32283458c7b79cbe369552bb82" exitCode=0 Jul 22 20:02:14.224773 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:14.224355 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" event={"ID":"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f","Type":"ContainerDied","Data":"dee57af68bd1651fb325caede114e960cbfc4a32283458c7b79cbe369552bb82"} Jul 22 20:02:14.496069 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:14.495971 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:14.496069 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:14.496050 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:14.496269 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:14.496081 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:14.496269 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:14.496131 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 20:02:14.496269 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:14.496151 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579b98c4-8z9rg: secret "image-registry-tls" not found Jul 22 20:02:14.496269 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:14.496177 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 20:02:14.496269 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:14.496205 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 20:02:14.496269 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:14.496222 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls podName:da2a3c08-e156-44fb-8571-5c3add922c89 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:22.496206751 +0000 UTC m=+52.052950424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls") pod "image-registry-579b98c4-8z9rg" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89") : secret "image-registry-tls" not found Jul 22 20:02:14.496269 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:14.496235 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls podName:c5545d45-8b89-4f48-813c-6c73a905c56a nodeName:}" failed. No retries permitted until 2026-07-22 20:02:22.49622957 +0000 UTC m=+52.052973243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls") pod "dns-default-hqpcf" (UID: "c5545d45-8b89-4f48-813c-6c73a905c56a") : secret "dns-default-metrics-tls" not found Jul 22 20:02:14.496269 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:14.496245 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert podName:b0f4e343-fae7-43f2-9cd7-15ea347781ad nodeName:}" failed. No retries permitted until 2026-07-22 20:02:22.496240184 +0000 UTC m=+52.052983856 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-r7mrz" (UID: "b0f4e343-fae7-43f2-9cd7-15ea347781ad") : secret "networking-console-plugin-cert" not found Jul 22 20:02:15.230299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:15.230266 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" event={"ID":"e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f","Type":"ContainerStarted","Data":"ed4f6297d11c8e29c8ec2c9afca2897d126b6ff4e1e3420ea2a7c4454c6c0dc4"} Jul 22 20:02:15.251538 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:15.251485 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ss5ld" podStartSLOduration=6.390817296 podStartE2EDuration="43.251465758s" totalImagesPullingTime="36.860648462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:02:15.250482289 +0000 UTC m=+44.807225985" watchObservedRunningTime="2026-07-22 20:02:15.251465758 +0000 UTC m=+44.808209452" Jul 22 20:02:22.564763 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:22.564720 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:22.565306 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:22.564791 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:22.565306 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:22.564842 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:22.565306 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:22.564910 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 20:02:22.565306 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:22.564953 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 20:02:22.565306 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:22.564977 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 20:02:22.565306 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:22.564990 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579b98c4-8z9rg: secret "image-registry-tls" not found Jul 22 20:02:22.565306 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:22.565016 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert podName:b0f4e343-fae7-43f2-9cd7-15ea347781ad nodeName:}" failed. No retries permitted until 2026-07-22 20:02:38.564991293 +0000 UTC m=+68.121734966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-r7mrz" (UID: "b0f4e343-fae7-43f2-9cd7-15ea347781ad") : secret "networking-console-plugin-cert" not found Jul 22 20:02:22.565306 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:22.565039 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls podName:c5545d45-8b89-4f48-813c-6c73a905c56a nodeName:}" failed. No retries permitted until 2026-07-22 20:02:38.56502893 +0000 UTC m=+68.121772610 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls") pod "dns-default-hqpcf" (UID: "c5545d45-8b89-4f48-813c-6c73a905c56a") : secret "dns-default-metrics-tls" not found Jul 22 20:02:22.565306 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:22.565055 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls podName:da2a3c08-e156-44fb-8571-5c3add922c89 nodeName:}" failed. No retries permitted until 2026-07-22 20:02:38.565045646 +0000 UTC m=+68.121789319 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls") pod "image-registry-579b98c4-8z9rg" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89") : secret "image-registry-tls" not found Jul 22 20:02:32.189929 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:32.189898 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xpshq" Jul 22 20:02:36.773870 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:36.773830 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:02:36.774362 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:36.773935 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:02:36.776305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:36.776278 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 20:02:36.776436 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:36.776364 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 20:02:36.785091 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:36.785063 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 20:02:36.785177 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:36.785166 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:03:40.78514331 +0000 UTC m=+130.341886994 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : secret "metrics-daemon-secret" not found Jul 22 20:02:36.789882 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:36.789849 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/74d38ca2-5c25-4236-a7b5-335fc07b9ca2-original-pull-secret\") pod \"global-pull-secret-syncer-mxfbh\" (UID: \"74d38ca2-5c25-4236-a7b5-335fc07b9ca2\") " pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:02:36.874459 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:36.874390 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:36.876672 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:36.876649 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 20:02:36.887251 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:36.887222 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 20:02:36.898473 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:36.898434 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgggl\" (UniqueName: \"kubernetes.io/projected/d47f328b-a9d1-45b5-b78e-a2b31af68583-kube-api-access-fgggl\") pod \"network-check-target-5dll9\" (UID: \"d47f328b-a9d1-45b5-b78e-a2b31af68583\") " pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:37.020270 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:37.020236 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nrd5t\"" Jul 22 20:02:37.022995 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:37.022961 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mxfbh" Jul 22 20:02:37.028554 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:37.028530 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:37.177019 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:37.176987 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mxfbh"] Jul 22 20:02:37.180343 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:02:37.180308 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d38ca2_5c25_4236_a7b5_335fc07b9ca2.slice/crio-ce89c3eed4cd7b095f3058b82f2576075c75f5bd6a51cf35a0d83be340c38c21 WatchSource:0}: Error finding container ce89c3eed4cd7b095f3058b82f2576075c75f5bd6a51cf35a0d83be340c38c21: Status 404 returned error can't find the container with id ce89c3eed4cd7b095f3058b82f2576075c75f5bd6a51cf35a0d83be340c38c21 Jul 22 20:02:37.192350 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:37.192315 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5dll9"] Jul 22 20:02:37.195569 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:02:37.195537 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47f328b_a9d1_45b5_b78e_a2b31af68583.slice/crio-59bce2283b3de34b36b920aeb27b4ff4c1a40123cf9d17ec0c86cca4ca72b4fb WatchSource:0}: Error finding container 59bce2283b3de34b36b920aeb27b4ff4c1a40123cf9d17ec0c86cca4ca72b4fb: Status 404 returned error can't find the container with id 59bce2283b3de34b36b920aeb27b4ff4c1a40123cf9d17ec0c86cca4ca72b4fb Jul 22 20:02:37.282902 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:37.282786 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mxfbh" event={"ID":"74d38ca2-5c25-4236-a7b5-335fc07b9ca2","Type":"ContainerStarted","Data":"ce89c3eed4cd7b095f3058b82f2576075c75f5bd6a51cf35a0d83be340c38c21"} Jul 22 20:02:37.283944 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:37.283917 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5dll9" event={"ID":"d47f328b-a9d1-45b5-b78e-a2b31af68583","Type":"ContainerStarted","Data":"59bce2283b3de34b36b920aeb27b4ff4c1a40123cf9d17ec0c86cca4ca72b4fb"} Jul 22 20:02:38.587448 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:38.587392 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:02:38.587913 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:38.587465 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:02:38.587913 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:38.587543 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:02:38.587913 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:38.587560 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 20:02:38.587913 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:38.587626 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls podName:c5545d45-8b89-4f48-813c-6c73a905c56a nodeName:}" failed. No retries permitted until 2026-07-22 20:03:10.58760499 +0000 UTC m=+100.144348686 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls") pod "dns-default-hqpcf" (UID: "c5545d45-8b89-4f48-813c-6c73a905c56a") : secret "dns-default-metrics-tls" not found Jul 22 20:02:38.587913 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:38.587633 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 20:02:38.587913 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:38.587647 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 20:02:38.587913 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:38.587675 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579b98c4-8z9rg: secret "image-registry-tls" not found Jul 22 20:02:38.587913 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:38.587677 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert podName:b0f4e343-fae7-43f2-9cd7-15ea347781ad nodeName:}" failed. No retries permitted until 2026-07-22 20:03:10.587667591 +0000 UTC m=+100.144411267 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-r7mrz" (UID: "b0f4e343-fae7-43f2-9cd7-15ea347781ad") : secret "networking-console-plugin-cert" not found Jul 22 20:02:38.587913 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:02:38.587744 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls podName:da2a3c08-e156-44fb-8571-5c3add922c89 nodeName:}" failed. No retries permitted until 2026-07-22 20:03:10.587723563 +0000 UTC m=+100.144467458 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls") pod "image-registry-579b98c4-8z9rg" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89") : secret "image-registry-tls" not found Jul 22 20:02:42.298975 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:42.298928 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mxfbh" event={"ID":"74d38ca2-5c25-4236-a7b5-335fc07b9ca2","Type":"ContainerStarted","Data":"a756bb49dd1bbdcc1d03d07a9c3ca5bf7d2a77f9d761583953dc96945477e97c"} Jul 22 20:02:42.313867 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:42.313787 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mxfbh" podStartSLOduration=65.589553211 podStartE2EDuration="1m10.313764467s" totalImagesPullingTime="4.724211256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:02:42.312170073 +0000 UTC m=+71.868913769" watchObservedRunningTime="2026-07-22 20:02:42.313764467 +0000 UTC m=+71.870508161" Jul 22 20:02:46.310056 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:46.310021 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5dll9" event={"ID":"d47f328b-a9d1-45b5-b78e-a2b31af68583","Type":"ContainerStarted","Data":"93cbd13573a5fb00f9faec328f4177dd8c5e6a5decd9281ecca75b470fb46b2e"} Jul 22 20:02:46.310447 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:46.310163 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:02:46.324541 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:02:46.324483 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5dll9" podStartSLOduration=65.613997504 podStartE2EDuration="1m14.324468353s" totalImagesPullingTime="8.710470849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:02:46.323942778 +0000 UTC m=+75.880686472" watchObservedRunningTime="2026-07-22 20:02:46.324468353 +0000 UTC m=+75.881212048" Jul 22 20:03:10.636989 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:03:10.636826 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:03:10.636989 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:03:10.636882 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:03:10.636989 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:03:10.636904 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:03:10.637688 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:03:10.636992 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 20:03:10.637688 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:03:10.637002 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 20:03:10.637688 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:03:10.637024 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 20:03:10.637688 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:03:10.637037 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579b98c4-8z9rg: secret "image-registry-tls" not found Jul 22 20:03:10.637688 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:03:10.637077 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert podName:b0f4e343-fae7-43f2-9cd7-15ea347781ad nodeName:}" failed. No retries permitted until 2026-07-22 20:04:14.637058597 +0000 UTC m=+164.193802270 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-r7mrz" (UID: "b0f4e343-fae7-43f2-9cd7-15ea347781ad") : secret "networking-console-plugin-cert" not found Jul 22 20:03:10.637688 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:03:10.637094 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls podName:da2a3c08-e156-44fb-8571-5c3add922c89 nodeName:}" failed. No retries permitted until 2026-07-22 20:04:14.637086607 +0000 UTC m=+164.193830280 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls") pod "image-registry-579b98c4-8z9rg" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89") : secret "image-registry-tls" not found Jul 22 20:03:10.637688 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:03:10.637104 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls podName:c5545d45-8b89-4f48-813c-6c73a905c56a nodeName:}" failed. No retries permitted until 2026-07-22 20:04:14.637098324 +0000 UTC m=+164.193841996 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls") pod "dns-default-hqpcf" (UID: "c5545d45-8b89-4f48-813c-6c73a905c56a") : secret "dns-default-metrics-tls" not found Jul 22 20:03:17.314716 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:03:17.314686 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5dll9" Jul 22 20:03:40.864553 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:03:40.864499 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:03:40.865032 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:03:40.864646 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 20:03:40.865032 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:03:40.864712 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:05:42.864695828 +0000 UTC m=+252.421439501 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : secret "metrics-daemon-secret" not found Jul 22 20:04:09.732902 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:09.732850 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" podUID="b0f4e343-fae7-43f2-9cd7-15ea347781ad" Jul 22 20:04:09.743068 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:09.743028 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" podUID="da2a3c08-e156-44fb-8571-5c3add922c89" Jul 22 20:04:09.763225 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:09.763190 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hqpcf" podUID="c5545d45-8b89-4f48-813c-6c73a905c56a" Jul 22 20:04:10.012085 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:10.011975 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:04:10.511158 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:04:10.511123 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:04:10.511336 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:04:10.511128 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:04:10.511336 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:04:10.511128 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hqpcf" Jul 22 20:04:14.712151 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:04:14.712096 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:04:14.712151 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:04:14.712150 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:04:14.712696 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:04:14.712174 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:04:14.712696 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:14.712252 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 20:04:14.712696 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:14.712281 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 20:04:14.712696 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:14.712292 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-579b98c4-8z9rg: secret "image-registry-tls" not found Jul 22 20:04:14.712696 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:14.712304 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 20:04:14.712696 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:14.712334 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert podName:b0f4e343-fae7-43f2-9cd7-15ea347781ad nodeName:}" failed. No retries permitted until 2026-07-22 20:06:16.712316595 +0000 UTC m=+286.269060272 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-r7mrz" (UID: "b0f4e343-fae7-43f2-9cd7-15ea347781ad") : secret "networking-console-plugin-cert" not found Jul 22 20:04:14.712696 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:14.712349 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls podName:da2a3c08-e156-44fb-8571-5c3add922c89 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:16.712342956 +0000 UTC m=+286.269086629 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls") pod "image-registry-579b98c4-8z9rg" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89") : secret "image-registry-tls" not found Jul 22 20:04:14.712696 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:04:14.712360 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls podName:c5545d45-8b89-4f48-813c-6c73a905c56a nodeName:}" failed. No retries permitted until 2026-07-22 20:06:16.712354078 +0000 UTC m=+286.269097786 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls") pod "dns-default-hqpcf" (UID: "c5545d45-8b89-4f48-813c-6c73a905c56a") : secret "dns-default-metrics-tls" not found Jul 22 20:04:24.001128 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:04:24.001023 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:05:42.941876 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:42.941833 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:05:42.942378 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:42.942002 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 20:05:42.942378 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:42.942083 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs podName:b5419f3c-d6a5-4530-872d-08505f1c1c74 nodeName:}" failed. No retries permitted until 2026-07-22 20:07:44.942066619 +0000 UTC m=+374.498810291 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs") pod "network-metrics-daemon-4cnsx" (UID: "b5419f3c-d6a5-4530-872d-08505f1c1c74") : secret "metrics-daemon-secret" not found Jul 22 20:05:52.438612 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.438552 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d"] Jul 22 20:05:52.441881 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.441844 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d" Jul 22 20:05:52.444812 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.444784 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 20:05:52.444992 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.444916 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 20:05:52.444992 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.444922 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mhmnf\"" Jul 22 20:05:52.445286 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.445270 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p"] Jul 22 20:05:52.447394 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.447374 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-sfgrv"] Jul 22 20:05:52.447515 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.447497 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:05:52.449179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.449159 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-p5zmg"] Jul 22 20:05:52.449323 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.449271 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.450805 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.450772 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 22 20:05:52.450805 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.450790 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 22 20:05:52.450805 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.450798 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 22 20:05:52.451022 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.450867 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-x5p24\"" Jul 22 20:05:52.451109 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.451092 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.452178 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.452155 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d"] Jul 22 20:05:52.453340 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.453269 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 20:05:52.453340 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.453297 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 22 20:05:52.453340 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.453333 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 20:05:52.453679 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.453660 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 20:05:52.454078 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.454048 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 20:05:52.454318 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.454290 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 22 20:05:52.454505 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.454331 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rhq58\"" Jul 22 20:05:52.454866 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.454847 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 20:05:52.454961 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.454912 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-g97kk\"" Jul 22 20:05:52.455025 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.454862 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 20:05:52.459939 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.459908 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p"] Jul 22 20:05:52.460332 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.460307 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 20:05:52.461242 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.461217 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-sfgrv"] Jul 22 20:05:52.461765 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.461740 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 22 20:05:52.462593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.462574 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-p5zmg"] Jul 22 20:05:52.512641 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512596 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52d3ecf9-455b-427d-9759-ecfeace7e339-config\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.512641 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512649 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz77d\" (UniqueName: \"kubernetes.io/projected/52d3ecf9-455b-427d-9759-ecfeace7e339-kube-api-access-cz77d\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.512914 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512710 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5f315bbc-8593-4ead-9e7e-06b55dee57c5-snapshots\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.512914 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512761 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f315bbc-8593-4ead-9e7e-06b55dee57c5-serving-cert\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.512914 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512810 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7hp8\" (UniqueName: \"kubernetes.io/projected/00ec5360-2150-4da1-bd5f-d2494479bc7e-kube-api-access-s7hp8\") pod \"volume-data-source-validator-78cf4c6f6-d7x7d\" (UID: \"00ec5360-2150-4da1-bd5f-d2494479bc7e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d" Jul 22 20:05:52.512914 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512834 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52d3ecf9-455b-427d-9759-ecfeace7e339-trusted-ca\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.512914 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512856 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52d3ecf9-455b-427d-9759-ecfeace7e339-serving-cert\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.512914 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512889 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77gjc\" (UniqueName: \"kubernetes.io/projected/5f315bbc-8593-4ead-9e7e-06b55dee57c5-kube-api-access-77gjc\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.513172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512955 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f315bbc-8593-4ead-9e7e-06b55dee57c5-tmp\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.513172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.512981 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:05:52.513172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.513011 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f315bbc-8593-4ead-9e7e-06b55dee57c5-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.513172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.513043 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f315bbc-8593-4ead-9e7e-06b55dee57c5-service-ca-bundle\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.513172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.513069 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47984\" (UniqueName: \"kubernetes.io/projected/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-kube-api-access-47984\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:05:52.614083 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614046 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f315bbc-8593-4ead-9e7e-06b55dee57c5-tmp\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.614083 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614082 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:05:52.614581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614111 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f315bbc-8593-4ead-9e7e-06b55dee57c5-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.614581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614143 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f315bbc-8593-4ead-9e7e-06b55dee57c5-service-ca-bundle\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.614581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614203 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47984\" (UniqueName: \"kubernetes.io/projected/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-kube-api-access-47984\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:05:52.614581 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:52.614216 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 20:05:52.614581 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:52.614335 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls podName:79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6 nodeName:}" failed. No retries permitted until 2026-07-22 20:05:53.114310938 +0000 UTC m=+262.671054632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-5r89p" (UID: "79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6") : secret "samples-operator-tls" not found Jul 22 20:05:52.614581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614466 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52d3ecf9-455b-427d-9759-ecfeace7e339-config\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.614581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614510 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz77d\" (UniqueName: \"kubernetes.io/projected/52d3ecf9-455b-427d-9759-ecfeace7e339-kube-api-access-cz77d\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.614581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614557 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5f315bbc-8593-4ead-9e7e-06b55dee57c5-snapshots\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.614581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614585 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f315bbc-8593-4ead-9e7e-06b55dee57c5-serving-cert\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.615039 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614606 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f315bbc-8593-4ead-9e7e-06b55dee57c5-tmp\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.615039 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614646 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7hp8\" (UniqueName: \"kubernetes.io/projected/00ec5360-2150-4da1-bd5f-d2494479bc7e-kube-api-access-s7hp8\") pod \"volume-data-source-validator-78cf4c6f6-d7x7d\" (UID: \"00ec5360-2150-4da1-bd5f-d2494479bc7e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d" Jul 22 20:05:52.615039 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614671 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52d3ecf9-455b-427d-9759-ecfeace7e339-trusted-ca\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.615039 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614696 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52d3ecf9-455b-427d-9759-ecfeace7e339-serving-cert\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.615039 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614722 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77gjc\" (UniqueName: \"kubernetes.io/projected/5f315bbc-8593-4ead-9e7e-06b55dee57c5-kube-api-access-77gjc\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.615039 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.614968 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f315bbc-8593-4ead-9e7e-06b55dee57c5-service-ca-bundle\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.615344 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.615137 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f315bbc-8593-4ead-9e7e-06b55dee57c5-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.615344 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.615198 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5f315bbc-8593-4ead-9e7e-06b55dee57c5-snapshots\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.615344 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.615211 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52d3ecf9-455b-427d-9759-ecfeace7e339-config\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.615827 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.615800 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52d3ecf9-455b-427d-9759-ecfeace7e339-trusted-ca\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.617241 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.617223 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f315bbc-8593-4ead-9e7e-06b55dee57c5-serving-cert\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.617305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.617226 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52d3ecf9-455b-427d-9759-ecfeace7e339-serving-cert\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.624376 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.624336 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47984\" (UniqueName: \"kubernetes.io/projected/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-kube-api-access-47984\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:05:52.624622 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.624591 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7hp8\" (UniqueName: \"kubernetes.io/projected/00ec5360-2150-4da1-bd5f-d2494479bc7e-kube-api-access-s7hp8\") pod \"volume-data-source-validator-78cf4c6f6-d7x7d\" (UID: \"00ec5360-2150-4da1-bd5f-d2494479bc7e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d" Jul 22 20:05:52.626925 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.626891 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz77d\" (UniqueName: \"kubernetes.io/projected/52d3ecf9-455b-427d-9759-ecfeace7e339-kube-api-access-cz77d\") pod \"console-operator-565b7b569d-p5zmg\" (UID: \"52d3ecf9-455b-427d-9759-ecfeace7e339\") " pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.627073 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.626943 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77gjc\" (UniqueName: \"kubernetes.io/projected/5f315bbc-8593-4ead-9e7e-06b55dee57c5-kube-api-access-77gjc\") pod \"insights-operator-555d9d7885-sfgrv\" (UID: \"5f315bbc-8593-4ead-9e7e-06b55dee57c5\") " pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.754112 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.754052 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d" Jul 22 20:05:52.770292 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.770252 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-sfgrv" Jul 22 20:05:52.775568 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.775524 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:05:52.912008 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.911871 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d"] Jul 22 20:05:52.916771 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:05:52.916738 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ec5360_2150_4da1_bd5f_d2494479bc7e.slice/crio-10cafb97637ab6e7e26087013a447dc5ac24a33bbde34e70f8c583d15d5db380 WatchSource:0}: Error finding container 10cafb97637ab6e7e26087013a447dc5ac24a33bbde34e70f8c583d15d5db380: Status 404 returned error can't find the container with id 10cafb97637ab6e7e26087013a447dc5ac24a33bbde34e70f8c583d15d5db380 Jul 22 20:05:52.930591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.930555 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-sfgrv"] Jul 22 20:05:52.933627 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:05:52.933595 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f315bbc_8593_4ead_9e7e_06b55dee57c5.slice/crio-49489b26396a900dc80f35799aa27c93e6b1564ef015a49b1997f4a3f589b70e WatchSource:0}: Error finding container 49489b26396a900dc80f35799aa27c93e6b1564ef015a49b1997f4a3f589b70e: Status 404 returned error can't find the container with id 49489b26396a900dc80f35799aa27c93e6b1564ef015a49b1997f4a3f589b70e Jul 22 20:05:52.948365 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:52.948324 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-p5zmg"] Jul 22 20:05:52.952004 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:05:52.951960 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d3ecf9_455b_427d_9759_ecfeace7e339.slice/crio-0fe781ebf1dc56077a323acbb19ac2e236ca276221ae1260c32429e879c7d00e WatchSource:0}: Error finding container 0fe781ebf1dc56077a323acbb19ac2e236ca276221ae1260c32429e879c7d00e: Status 404 returned error can't find the container with id 0fe781ebf1dc56077a323acbb19ac2e236ca276221ae1260c32429e879c7d00e Jul 22 20:05:53.119953 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:53.119851 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:05:53.120102 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:53.120051 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 20:05:53.120142 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:53.120122 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls podName:79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6 nodeName:}" failed. No retries permitted until 2026-07-22 20:05:54.120104757 +0000 UTC m=+263.676848439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-5r89p" (UID: "79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6") : secret "samples-operator-tls" not found Jul 22 20:05:53.764850 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:53.764806 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" event={"ID":"52d3ecf9-455b-427d-9759-ecfeace7e339","Type":"ContainerStarted","Data":"0fe781ebf1dc56077a323acbb19ac2e236ca276221ae1260c32429e879c7d00e"} Jul 22 20:05:53.766353 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:53.766306 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sfgrv" event={"ID":"5f315bbc-8593-4ead-9e7e-06b55dee57c5","Type":"ContainerStarted","Data":"49489b26396a900dc80f35799aa27c93e6b1564ef015a49b1997f4a3f589b70e"} Jul 22 20:05:53.767883 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:53.767846 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d" event={"ID":"00ec5360-2150-4da1-bd5f-d2494479bc7e","Type":"ContainerStarted","Data":"10cafb97637ab6e7e26087013a447dc5ac24a33bbde34e70f8c583d15d5db380"} Jul 22 20:05:54.130008 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:54.129903 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:05:54.130187 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:54.130087 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 20:05:54.130187 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:54.130169 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls podName:79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6 nodeName:}" failed. No retries permitted until 2026-07-22 20:05:56.13015106 +0000 UTC m=+265.686894737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-5r89p" (UID: "79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6") : secret "samples-operator-tls" not found Jul 22 20:05:55.775050 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:55.775007 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sfgrv" event={"ID":"5f315bbc-8593-4ead-9e7e-06b55dee57c5","Type":"ContainerStarted","Data":"8ef75c4193c329ead671ca81119abce9e1d5bd354f1dfe71e2212bf64675ef09"} Jul 22 20:05:55.776647 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:55.776617 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d" event={"ID":"00ec5360-2150-4da1-bd5f-d2494479bc7e","Type":"ContainerStarted","Data":"78cc7368ee94509d17b4b1eb1c2c6c89482f383d0c851aab159804a48a50cc49"} Jul 22 20:05:55.791178 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:55.791117 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-sfgrv" podStartSLOduration=1.843255037 podStartE2EDuration="3.791099318s" totalImagesPullingTime="1.947844281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:05:55.790316945 +0000 UTC m=+265.347060641" watchObservedRunningTime="2026-07-22 20:05:55.791099318 +0000 UTC m=+265.347843106" Jul 22 20:05:55.803875 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:55.803814 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-d7x7d" podStartSLOduration=1.843799364 podStartE2EDuration="3.803798029s" totalImagesPullingTime="1.959998665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:05:55.802489699 +0000 UTC m=+265.359233399" watchObservedRunningTime="2026-07-22 20:05:55.803798029 +0000 UTC m=+265.360541723" Jul 22 20:05:56.150479 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.150352 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:05:56.150665 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:56.150546 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 20:05:56.150665 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:56.150636 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls podName:79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:00.150612449 +0000 UTC m=+269.707356137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-5r89p" (UID: "79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6") : secret "samples-operator-tls" not found Jul 22 20:05:56.323233 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.323200 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zd8zj"] Jul 22 20:05:56.326288 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.326264 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.328872 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.328838 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 20:05:56.329034 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.328853 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 20:05:56.330044 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.330022 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tfrxf\"" Jul 22 20:05:56.342517 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.342481 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zd8zj"] Jul 22 20:05:56.452979 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.452888 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmxqz\" (UniqueName: \"kubernetes.io/projected/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-api-access-qmxqz\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.453126 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.453012 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.453126 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.453055 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.453126 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.453086 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0458ea69-7f6c-4806-a27c-adaf0d15ad76-crio-socket\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.453228 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.453145 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0458ea69-7f6c-4806-a27c-adaf0d15ad76-data-volume\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.553784 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.553741 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.553993 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.553793 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.553993 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.553825 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0458ea69-7f6c-4806-a27c-adaf0d15ad76-crio-socket\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.553993 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.553871 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0458ea69-7f6c-4806-a27c-adaf0d15ad76-data-volume\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.553993 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:56.553871 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 20:05:56.553993 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.553913 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmxqz\" (UniqueName: \"kubernetes.io/projected/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-api-access-qmxqz\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.553993 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:56.553924 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 20:05:56.553993 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:56.553966 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:05:57.053943921 +0000 UTC m=+266.610687596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : configmap "kube-rbac-proxy" not found Jul 22 20:05:56.553993 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:56.553992 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:05:57.05397467 +0000 UTC m=+266.610718349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : secret "insights-runtime-extractor-tls" not found Jul 22 20:05:56.554391 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.554170 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0458ea69-7f6c-4806-a27c-adaf0d15ad76-crio-socket\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.554507 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.554485 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0458ea69-7f6c-4806-a27c-adaf0d15ad76-data-volume\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.564482 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.564440 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmxqz\" (UniqueName: \"kubernetes.io/projected/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-api-access-qmxqz\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:56.781377 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.781348 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/0.log" Jul 22 20:05:56.781810 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.781393 2553 generic.go:356] "Generic (PLEG): container finished" podID="52d3ecf9-455b-427d-9759-ecfeace7e339" containerID="d5973b4019a18d10e39f9f856e102de162ba5e57c7d1799a6fda1f47f179f3d0" exitCode=255 Jul 22 20:05:56.781810 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.781462 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" event={"ID":"52d3ecf9-455b-427d-9759-ecfeace7e339","Type":"ContainerDied","Data":"d5973b4019a18d10e39f9f856e102de162ba5e57c7d1799a6fda1f47f179f3d0"} Jul 22 20:05:56.781810 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:56.781738 2553 scope.go:117] "RemoveContainer" containerID="d5973b4019a18d10e39f9f856e102de162ba5e57c7d1799a6fda1f47f179f3d0" Jul 22 20:05:57.058915 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:57.058811 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:57.058915 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:57.058857 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:57.059111 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:57.058944 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 20:05:57.059111 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:57.059009 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 20:05:57.059111 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:57.059011 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:05:58.058995487 +0000 UTC m=+267.615739159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : configmap "kube-rbac-proxy" not found Jul 22 20:05:57.059111 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:57.059063 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:05:58.059047995 +0000 UTC m=+267.615791672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : secret "insights-runtime-extractor-tls" not found Jul 22 20:05:57.786142 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:57.786108 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:05:57.786700 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:57.786606 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/0.log" Jul 22 20:05:57.786700 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:57.786644 2553 generic.go:356] "Generic (PLEG): container finished" podID="52d3ecf9-455b-427d-9759-ecfeace7e339" containerID="3730bbad24c9ad9b1cd078936e8d3e581e658bb8399393b8032ee3a4835c3001" exitCode=255 Jul 22 20:05:57.786788 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:57.786716 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" event={"ID":"52d3ecf9-455b-427d-9759-ecfeace7e339","Type":"ContainerDied","Data":"3730bbad24c9ad9b1cd078936e8d3e581e658bb8399393b8032ee3a4835c3001"} Jul 22 20:05:57.786788 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:57.786769 2553 scope.go:117] "RemoveContainer" containerID="d5973b4019a18d10e39f9f856e102de162ba5e57c7d1799a6fda1f47f179f3d0" Jul 22 20:05:57.786995 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:57.786969 2553 scope.go:117] "RemoveContainer" containerID="3730bbad24c9ad9b1cd078936e8d3e581e658bb8399393b8032ee3a4835c3001" Jul 22 20:05:57.787216 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:57.787190 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-p5zmg_openshift-console-operator(52d3ecf9-455b-427d-9759-ecfeace7e339)\"" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" podUID="52d3ecf9-455b-427d-9759-ecfeace7e339" Jul 22 20:05:58.068572 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.068488 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:58.068572 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.068527 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:05:58.068786 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:58.068602 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 20:05:58.068786 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:58.068650 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 20:05:58.068786 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:58.068666 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:00.068646979 +0000 UTC m=+269.625390665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : configmap "kube-rbac-proxy" not found Jul 22 20:05:58.068786 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:58.068689 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:00.068678634 +0000 UTC m=+269.625422307 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : secret "insights-runtime-extractor-tls" not found Jul 22 20:05:58.177271 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.177235 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t"] Jul 22 20:05:58.179854 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.179835 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t" Jul 22 20:05:58.182143 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.182079 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 22 20:05:58.182143 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.182114 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-snp25\"" Jul 22 20:05:58.182325 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.182248 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 22 20:05:58.189715 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.189689 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t"] Jul 22 20:05:58.270966 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.270910 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f49xh\" (UniqueName: \"kubernetes.io/projected/4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc-kube-api-access-f49xh\") pod \"migrator-7cc9c7ffd7-nf56t\" (UID: \"4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t" Jul 22 20:05:58.371499 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.371367 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f49xh\" (UniqueName: \"kubernetes.io/projected/4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc-kube-api-access-f49xh\") pod \"migrator-7cc9c7ffd7-nf56t\" (UID: \"4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t" Jul 22 20:05:58.379165 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.379136 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f49xh\" (UniqueName: \"kubernetes.io/projected/4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc-kube-api-access-f49xh\") pod \"migrator-7cc9c7ffd7-nf56t\" (UID: \"4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t" Jul 22 20:05:58.490231 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.490185 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t" Jul 22 20:05:58.502253 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.502229 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zq625_4da17f54-06b4-4750-9f1b-3304f6686447/dns-node-resolver/0.log" Jul 22 20:05:58.625248 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.625120 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t"] Jul 22 20:05:58.628958 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:05:58.628918 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b5f2c58_24d0_4ba1_a4f3_eaa4a132ccfc.slice/crio-d9c2ec890610c6ded13999f1905d30034b6672a65a96f7a3e7ba7ebfecf5cb73 WatchSource:0}: Error finding container d9c2ec890610c6ded13999f1905d30034b6672a65a96f7a3e7ba7ebfecf5cb73: Status 404 returned error can't find the container with id d9c2ec890610c6ded13999f1905d30034b6672a65a96f7a3e7ba7ebfecf5cb73 Jul 22 20:05:58.791209 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.791181 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:05:58.791664 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.791580 2553 scope.go:117] "RemoveContainer" containerID="3730bbad24c9ad9b1cd078936e8d3e581e658bb8399393b8032ee3a4835c3001" Jul 22 20:05:58.791797 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:05:58.791776 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-p5zmg_openshift-console-operator(52d3ecf9-455b-427d-9759-ecfeace7e339)\"" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" podUID="52d3ecf9-455b-427d-9759-ecfeace7e339" Jul 22 20:05:58.792454 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:58.792433 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t" event={"ID":"4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc","Type":"ContainerStarted","Data":"d9c2ec890610c6ded13999f1905d30034b6672a65a96f7a3e7ba7ebfecf5cb73"} Jul 22 20:05:59.501834 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:59.501808 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rbd5p_ad9034e0-a2b0-4bb7-bef5-b124b545dc59/node-ca/0.log" Jul 22 20:05:59.902572 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:05:59.902459 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5g5zk_8c525512-dbbd-4e74-8b1e-28270cfb6554/serve-healthcheck-canary/0.log" Jul 22 20:06:00.087558 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.087517 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:00.087558 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.087559 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:00.087787 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:00.087654 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 20:06:00.087787 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:00.087677 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 20:06:00.087787 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:00.087728 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:04.08771434 +0000 UTC m=+273.644458013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : secret "insights-runtime-extractor-tls" not found Jul 22 20:06:00.087787 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:00.087740 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:04.087733978 +0000 UTC m=+273.644477650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : configmap "kube-rbac-proxy" not found Jul 22 20:06:00.188675 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.188574 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:06:00.188829 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:00.188761 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 20:06:00.188877 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:00.188847 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls podName:79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:08.188830233 +0000 UTC m=+277.745573906 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-5r89p" (UID: "79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6") : secret "samples-operator-tls" not found Jul 22 20:06:00.684241 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.684176 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-2rgmx"] Jul 22 20:06:00.687674 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.687636 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.690310 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.690265 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 20:06:00.691001 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.690967 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 20:06:00.691001 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.690967 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 20:06:00.691296 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.690967 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 20:06:00.691296 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.690967 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ghlwz\"" Jul 22 20:06:00.698898 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.698870 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-2rgmx"] Jul 22 20:06:00.793239 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.793190 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tn6q\" (UniqueName: \"kubernetes.io/projected/a50d882c-ca0c-4b2e-826f-b626637d6a19-kube-api-access-4tn6q\") pod \"service-ca-58598695f7-2rgmx\" (UID: \"a50d882c-ca0c-4b2e-826f-b626637d6a19\") " pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.793459 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.793289 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a50d882c-ca0c-4b2e-826f-b626637d6a19-signing-cabundle\") pod \"service-ca-58598695f7-2rgmx\" (UID: \"a50d882c-ca0c-4b2e-826f-b626637d6a19\") " pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.793459 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.793373 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a50d882c-ca0c-4b2e-826f-b626637d6a19-signing-key\") pod \"service-ca-58598695f7-2rgmx\" (UID: \"a50d882c-ca0c-4b2e-826f-b626637d6a19\") " pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.894400 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.894362 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tn6q\" (UniqueName: \"kubernetes.io/projected/a50d882c-ca0c-4b2e-826f-b626637d6a19-kube-api-access-4tn6q\") pod \"service-ca-58598695f7-2rgmx\" (UID: \"a50d882c-ca0c-4b2e-826f-b626637d6a19\") " pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.894603 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.894546 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a50d882c-ca0c-4b2e-826f-b626637d6a19-signing-cabundle\") pod \"service-ca-58598695f7-2rgmx\" (UID: \"a50d882c-ca0c-4b2e-826f-b626637d6a19\") " pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.894661 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.894649 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a50d882c-ca0c-4b2e-826f-b626637d6a19-signing-key\") pod \"service-ca-58598695f7-2rgmx\" (UID: \"a50d882c-ca0c-4b2e-826f-b626637d6a19\") " pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.895385 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.895333 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a50d882c-ca0c-4b2e-826f-b626637d6a19-signing-cabundle\") pod \"service-ca-58598695f7-2rgmx\" (UID: \"a50d882c-ca0c-4b2e-826f-b626637d6a19\") " pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.897196 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.897172 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a50d882c-ca0c-4b2e-826f-b626637d6a19-signing-key\") pod \"service-ca-58598695f7-2rgmx\" (UID: \"a50d882c-ca0c-4b2e-826f-b626637d6a19\") " pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.904532 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.904500 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tn6q\" (UniqueName: \"kubernetes.io/projected/a50d882c-ca0c-4b2e-826f-b626637d6a19-kube-api-access-4tn6q\") pod \"service-ca-58598695f7-2rgmx\" (UID: \"a50d882c-ca0c-4b2e-826f-b626637d6a19\") " pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:00.999987 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:00.999935 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-2rgmx" Jul 22 20:06:01.163143 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:01.163107 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-2rgmx"] Jul 22 20:06:01.166991 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:01.166952 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50d882c_ca0c_4b2e_826f_b626637d6a19.slice/crio-73163b65cc76079347f26c05a7386c1183636344475becb54e4c4b95200443b9 WatchSource:0}: Error finding container 73163b65cc76079347f26c05a7386c1183636344475becb54e4c4b95200443b9: Status 404 returned error can't find the container with id 73163b65cc76079347f26c05a7386c1183636344475becb54e4c4b95200443b9 Jul 22 20:06:01.804124 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:01.804047 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-2rgmx" event={"ID":"a50d882c-ca0c-4b2e-826f-b626637d6a19","Type":"ContainerStarted","Data":"73163b65cc76079347f26c05a7386c1183636344475becb54e4c4b95200443b9"} Jul 22 20:06:02.776124 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:02.776063 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:06:02.776124 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:02.776141 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:06:02.776689 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:02.776619 2553 scope.go:117] "RemoveContainer" containerID="3730bbad24c9ad9b1cd078936e8d3e581e658bb8399393b8032ee3a4835c3001" Jul 22 20:06:02.776858 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:02.776827 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-p5zmg_openshift-console-operator(52d3ecf9-455b-427d-9759-ecfeace7e339)\"" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" podUID="52d3ecf9-455b-427d-9759-ecfeace7e339" Jul 22 20:06:02.811504 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:02.811461 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t" event={"ID":"4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc","Type":"ContainerStarted","Data":"9ee4533aa5918d85cd354b702efec41ff8568c0267f2f3db63aa88002b2dbc97"} Jul 22 20:06:02.811668 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:02.811526 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t" event={"ID":"4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc","Type":"ContainerStarted","Data":"d76e06caf5bd9814ba387ef614cd0dabfae25958efa1d348ea326abc5051bc2a"} Jul 22 20:06:02.827133 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:02.827073 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nf56t" podStartSLOduration=1.6140433299999999 podStartE2EDuration="4.827056926s" totalImagesPullingTime="3.213013596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:02.826442813 +0000 UTC m=+272.383186509" watchObservedRunningTime="2026-07-22 20:06:02.827056926 +0000 UTC m=+272.383800621" Jul 22 20:06:03.817730 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:03.817682 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-2rgmx" event={"ID":"a50d882c-ca0c-4b2e-826f-b626637d6a19","Type":"ContainerStarted","Data":"92296092bad948fbcbf04b9bcc6485ea8f0d0d2599b6a6a7b6d19dd640c32737"} Jul 22 20:06:03.833870 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:03.833804 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-2rgmx" podStartSLOduration=1.9174625650000001 podStartE2EDuration="3.833783315s" totalImagesPullingTime="1.91632075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:03.83283823 +0000 UTC m=+273.389581926" watchObservedRunningTime="2026-07-22 20:06:03.833783315 +0000 UTC m=+273.390527056" Jul 22 20:06:04.127187 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:04.127069 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:04.127187 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:04.127116 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:04.127460 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:04.127213 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 20:06:04.127460 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:04.127255 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 20:06:04.127460 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:04.127283 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:12.127266196 +0000 UTC m=+281.684009872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : configmap "kube-rbac-proxy" not found Jul 22 20:06:04.127460 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:04.127303 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:12.127292045 +0000 UTC m=+281.684035719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : secret "insights-runtime-extractor-tls" not found Jul 22 20:06:08.266711 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:08.266658 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:06:08.269330 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:08.269300 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-5r89p\" (UID: \"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:06:08.362024 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:08.361969 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" Jul 22 20:06:08.503536 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:08.503491 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p"] Jul 22 20:06:08.835850 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:08.835762 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" event={"ID":"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6","Type":"ContainerStarted","Data":"5b7c553abb906807a2c0a5296e54eec71718e82d7845fb901ec69a9c1c6bfc02"} Jul 22 20:06:10.844968 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:10.844876 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" event={"ID":"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6","Type":"ContainerStarted","Data":"49f512c2c3653a3070fb2028f83ed89b9b4dd4028ed9e6fce83e97595eb9da33"} Jul 22 20:06:10.844968 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:10.844931 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" event={"ID":"79392a8a-6de9-4b41-a7a1-6e0d9c3f1fd6","Type":"ContainerStarted","Data":"9b5009be3e806536c082e6dde8a995ae34dd4513c379088c716e61b8ce29ff60"} Jul 22 20:06:10.864843 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:10.864757 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-5r89p" podStartSLOduration=16.830890597 podStartE2EDuration="18.864736479s" totalImagesPullingTime="2.033845882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:10.863697526 +0000 UTC m=+280.420441237" watchObservedRunningTime="2026-07-22 20:06:10.864736479 +0000 UTC m=+280.421480175" Jul 22 20:06:12.202037 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:12.201980 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:12.202037 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:12.202038 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:12.202522 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:12.202155 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 20:06:12.202522 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:12.202232 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls podName:0458ea69-7f6c-4806-a27c-adaf0d15ad76 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:28.202212757 +0000 UTC m=+297.758956430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zd8zj" (UID: "0458ea69-7f6c-4806-a27c-adaf0d15ad76") : secret "insights-runtime-extractor-tls" not found Jul 22 20:06:12.203368 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:12.203338 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0458ea69-7f6c-4806-a27c-adaf0d15ad76-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:13.512346 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:13.512289 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" podUID="b0f4e343-fae7-43f2-9cd7-15ea347781ad" Jul 22 20:06:13.512346 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:13.512321 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hqpcf" podUID="c5545d45-8b89-4f48-813c-6c73a905c56a" Jul 22 20:06:13.512872 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:13.512402 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" podUID="da2a3c08-e156-44fb-8571-5c3add922c89" Jul 22 20:06:13.854531 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:13.854434 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:06:13.854754 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:13.854444 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hqpcf" Jul 22 20:06:13.854754 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:13.854442 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:06:16.001326 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.001258 2553 scope.go:117] "RemoveContainer" containerID="3730bbad24c9ad9b1cd078936e8d3e581e658bb8399393b8032ee3a4835c3001" Jul 22 20:06:16.744324 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.744282 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:06:16.744324 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.744327 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:06:16.744577 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.744379 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:06:16.747042 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.747011 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5545d45-8b89-4f48-813c-6c73a905c56a-metrics-tls\") pod \"dns-default-hqpcf\" (UID: \"c5545d45-8b89-4f48-813c-6c73a905c56a\") " pod="openshift-dns/dns-default-hqpcf" Jul 22 20:06:16.747189 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.747056 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0f4e343-fae7-43f2-9cd7-15ea347781ad-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-r7mrz\" (UID: \"b0f4e343-fae7-43f2-9cd7-15ea347781ad\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:06:16.747189 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.747069 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"image-registry-579b98c4-8z9rg\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:06:16.858610 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.858572 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zbn42\"" Jul 22 20:06:16.858610 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.858579 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-n9xf4\"" Jul 22 20:06:16.858842 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.858652 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gzjk5\"" Jul 22 20:06:16.865738 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.865702 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:06:16.865974 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.865757 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hqpcf" Jul 22 20:06:16.865974 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.865769 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" Jul 22 20:06:16.868699 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.868680 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:06:16.868754 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.868733 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" event={"ID":"52d3ecf9-455b-427d-9759-ecfeace7e339","Type":"ContainerStarted","Data":"f92b2e59c3083015d85442af912a32eb41387ce8c7b8a2c4dff7be678fede55f"} Jul 22 20:06:16.869158 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.869126 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:06:16.875293 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.875260 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" Jul 22 20:06:16.886832 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:16.886479 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-p5zmg" podStartSLOduration=21.375182784 podStartE2EDuration="24.886453663s" totalImagesPullingTime="3.511270879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:16.885966262 +0000 UTC m=+286.442709992" watchObservedRunningTime="2026-07-22 20:06:16.886453663 +0000 UTC m=+286.443197359" Jul 22 20:06:17.045055 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:17.045013 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-579b98c4-8z9rg"] Jul 22 20:06:17.060075 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:17.060014 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2a3c08_e156_44fb_8571_5c3add922c89.slice/crio-7e08aa087f1b07e4c8713de075d02ba9b9962907395515309b927b077e1ff5ef WatchSource:0}: Error finding container 7e08aa087f1b07e4c8713de075d02ba9b9962907395515309b927b077e1ff5ef: Status 404 returned error can't find the container with id 7e08aa087f1b07e4c8713de075d02ba9b9962907395515309b927b077e1ff5ef Jul 22 20:06:17.077761 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:17.077738 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz"] Jul 22 20:06:17.080350 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:17.080314 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f4e343_fae7_43f2_9cd7_15ea347781ad.slice/crio-4249b3ee178def27e8f60abdc6afe1cbc96387097f2bf3a4750b42e0f5661d64 WatchSource:0}: Error finding container 4249b3ee178def27e8f60abdc6afe1cbc96387097f2bf3a4750b42e0f5661d64: Status 404 returned error can't find the container with id 4249b3ee178def27e8f60abdc6afe1cbc96387097f2bf3a4750b42e0f5661d64 Jul 22 20:06:17.267419 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:17.267308 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hqpcf"] Jul 22 20:06:17.270828 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:17.270786 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5545d45_8b89_4f48_813c_6c73a905c56a.slice/crio-dedd61dcd9c7283df8d66b073b8bd85373174f8a1c6cb5ee5f188bf4902d2afb WatchSource:0}: Error finding container dedd61dcd9c7283df8d66b073b8bd85373174f8a1c6cb5ee5f188bf4902d2afb: Status 404 returned error can't find the container with id dedd61dcd9c7283df8d66b073b8bd85373174f8a1c6cb5ee5f188bf4902d2afb Jul 22 20:06:17.873722 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:17.873678 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hqpcf" event={"ID":"c5545d45-8b89-4f48-813c-6c73a905c56a","Type":"ContainerStarted","Data":"dedd61dcd9c7283df8d66b073b8bd85373174f8a1c6cb5ee5f188bf4902d2afb"} Jul 22 20:06:17.874932 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:17.874898 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" event={"ID":"b0f4e343-fae7-43f2-9cd7-15ea347781ad","Type":"ContainerStarted","Data":"4249b3ee178def27e8f60abdc6afe1cbc96387097f2bf3a4750b42e0f5661d64"} Jul 22 20:06:17.876400 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:17.876372 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" event={"ID":"da2a3c08-e156-44fb-8571-5c3add922c89","Type":"ContainerStarted","Data":"eee036901aa246eaec45891d353e47394fa757f40445fe44ac32c72d453a7a5d"} Jul 22 20:06:17.876551 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:17.876430 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" event={"ID":"da2a3c08-e156-44fb-8571-5c3add922c89","Type":"ContainerStarted","Data":"7e08aa087f1b07e4c8713de075d02ba9b9962907395515309b927b077e1ff5ef"} Jul 22 20:06:17.876614 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:17.876600 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:06:17.898498 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:17.898442 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" podStartSLOduration=285.898402804 podStartE2EDuration="4m45.898402804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:17.897728482 +0000 UTC m=+287.454472214" watchObservedRunningTime="2026-07-22 20:06:17.898402804 +0000 UTC m=+287.455146478" Jul 22 20:06:19.581151 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.581100 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-xhjxq"] Jul 22 20:06:19.584821 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.584772 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-xhjxq" Jul 22 20:06:19.587720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.587471 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-s7rn4\"" Jul 22 20:06:19.587720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.587498 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 20:06:19.587720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.587549 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 20:06:19.595670 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.594854 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-xhjxq"] Jul 22 20:06:19.685805 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.685665 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82bdb\" (UniqueName: \"kubernetes.io/projected/ca4b953d-3730-4b93-b12b-feb323a8b179-kube-api-access-82bdb\") pod \"downloads-5bc67dcd46-xhjxq\" (UID: \"ca4b953d-3730-4b93-b12b-feb323a8b179\") " pod="openshift-console/downloads-5bc67dcd46-xhjxq" Jul 22 20:06:19.787036 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.786983 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82bdb\" (UniqueName: \"kubernetes.io/projected/ca4b953d-3730-4b93-b12b-feb323a8b179-kube-api-access-82bdb\") pod \"downloads-5bc67dcd46-xhjxq\" (UID: \"ca4b953d-3730-4b93-b12b-feb323a8b179\") " pod="openshift-console/downloads-5bc67dcd46-xhjxq" Jul 22 20:06:19.807127 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.807055 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82bdb\" (UniqueName: \"kubernetes.io/projected/ca4b953d-3730-4b93-b12b-feb323a8b179-kube-api-access-82bdb\") pod \"downloads-5bc67dcd46-xhjxq\" (UID: \"ca4b953d-3730-4b93-b12b-feb323a8b179\") " pod="openshift-console/downloads-5bc67dcd46-xhjxq" Jul 22 20:06:19.887211 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.887166 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hqpcf" event={"ID":"c5545d45-8b89-4f48-813c-6c73a905c56a","Type":"ContainerStarted","Data":"6be2887815135f4e46230eab14c8f093af90714e74448929486c0fec8e24bddf"} Jul 22 20:06:19.887493 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.887236 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hqpcf" event={"ID":"c5545d45-8b89-4f48-813c-6c73a905c56a","Type":"ContainerStarted","Data":"24e7ea6dadd6a923f9ff760ef56f52a85cd216f3cd683a0380b0deb8ed9d1c01"} Jul 22 20:06:19.887493 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.887336 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hqpcf" Jul 22 20:06:19.904034 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.903986 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-xhjxq" Jul 22 20:06:19.915266 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:19.910780 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hqpcf" podStartSLOduration=251.764942452 podStartE2EDuration="4m13.91074261s" totalImagesPullingTime="2.145800158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:19.907745011 +0000 UTC m=+289.464488745" watchObservedRunningTime="2026-07-22 20:06:19.91074261 +0000 UTC m=+289.467486305" Jul 22 20:06:20.096260 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:20.096175 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-xhjxq"] Jul 22 20:06:20.102090 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:20.102018 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca4b953d_3730_4b93_b12b_feb323a8b179.slice/crio-39a3974b9a1e35b5513b8016dcccb5cd7692df44388f36c116b79ca8d6907928 WatchSource:0}: Error finding container 39a3974b9a1e35b5513b8016dcccb5cd7692df44388f36c116b79ca8d6907928: Status 404 returned error can't find the container with id 39a3974b9a1e35b5513b8016dcccb5cd7692df44388f36c116b79ca8d6907928 Jul 22 20:06:20.894900 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:20.894846 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-xhjxq" event={"ID":"ca4b953d-3730-4b93-b12b-feb323a8b179","Type":"ContainerStarted","Data":"39a3974b9a1e35b5513b8016dcccb5cd7692df44388f36c116b79ca8d6907928"} Jul 22 20:06:20.897671 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:20.897608 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" event={"ID":"b0f4e343-fae7-43f2-9cd7-15ea347781ad","Type":"ContainerStarted","Data":"bdcafd363fbc5b998022d88f141c0752b95d7d1126c7e38e246f27d33888b3d0"} Jul 22 20:06:20.915599 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:20.915512 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-r7mrz" podStartSLOduration=269.311959927 podStartE2EDuration="4m32.915489491s" totalImagesPullingTime="3.603529564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:20.914680506 +0000 UTC m=+290.471424262" watchObservedRunningTime="2026-07-22 20:06:20.915489491 +0000 UTC m=+290.472233187" Jul 22 20:06:25.596592 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.596556 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-848c569bfc-hqbqv"] Jul 22 20:06:25.599983 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.599957 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.602218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.602169 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 20:06:25.602337 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.602247 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 20:06:25.602920 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.602902 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 20:06:25.603164 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.603146 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 20:06:25.603252 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.603165 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pv4ng\"" Jul 22 20:06:25.603252 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.603148 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 20:06:25.609513 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.609491 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-848c569bfc-hqbqv"] Jul 22 20:06:25.739292 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.739248 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-config\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.739497 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.739301 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-service-ca\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.739497 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.739349 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pr6r\" (UniqueName: \"kubernetes.io/projected/d9a0989e-c780-4f1e-8de4-25a762111ee7-kube-api-access-5pr6r\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.739497 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.739397 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-oauth-config\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.739497 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.739472 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-oauth-serving-cert\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.739670 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.739543 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-serving-cert\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.840170 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.840127 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-oauth-config\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.840170 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.840174 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-oauth-serving-cert\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.840404 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.840202 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-serving-cert\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.840404 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.840233 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-config\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.840404 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.840254 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-service-ca\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.840404 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.840271 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pr6r\" (UniqueName: \"kubernetes.io/projected/d9a0989e-c780-4f1e-8de4-25a762111ee7-kube-api-access-5pr6r\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.841014 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.840990 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-service-ca\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.841109 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.841016 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-config\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.841109 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.841016 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-oauth-serving-cert\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.843060 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.843038 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-oauth-config\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.843228 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.843207 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-serving-cert\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.848163 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.848116 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pr6r\" (UniqueName: \"kubernetes.io/projected/d9a0989e-c780-4f1e-8de4-25a762111ee7-kube-api-access-5pr6r\") pod \"console-848c569bfc-hqbqv\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:25.912155 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:25.912119 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:26.056023 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:26.055910 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-848c569bfc-hqbqv"] Jul 22 20:06:26.059206 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:26.059175 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a0989e_c780_4f1e_8de4_25a762111ee7.slice/crio-ade91d56b70ec487b322c18afa5edbf1602e74aab397d7796b27e29136cee385 WatchSource:0}: Error finding container ade91d56b70ec487b322c18afa5edbf1602e74aab397d7796b27e29136cee385: Status 404 returned error can't find the container with id ade91d56b70ec487b322c18afa5edbf1602e74aab397d7796b27e29136cee385 Jul 22 20:06:26.922922 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:26.922877 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-848c569bfc-hqbqv" event={"ID":"d9a0989e-c780-4f1e-8de4-25a762111ee7","Type":"ContainerStarted","Data":"ade91d56b70ec487b322c18afa5edbf1602e74aab397d7796b27e29136cee385"} Jul 22 20:06:27.002379 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:27.002332 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4cnsx" podUID="b5419f3c-d6a5-4530-872d-08505f1c1c74" Jul 22 20:06:28.057072 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.057028 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2"] Jul 22 20:06:28.060510 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.060487 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" Jul 22 20:06:28.062971 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.062687 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 20:06:28.062971 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.062827 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jcg5l\"" Jul 22 20:06:28.069884 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.069841 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2"] Jul 22 20:06:28.160544 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.160484 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7080bba7-a4ad-4f85-bdd7-04c7b8539dc1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-p9xw2\" (UID: \"7080bba7-a4ad-4f85-bdd7-04c7b8539dc1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" Jul 22 20:06:28.261803 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.261755 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:28.261968 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.261833 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7080bba7-a4ad-4f85-bdd7-04c7b8539dc1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-p9xw2\" (UID: \"7080bba7-a4ad-4f85-bdd7-04c7b8539dc1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" Jul 22 20:06:28.261968 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:28.261947 2553 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 22 20:06:28.262062 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:28.262015 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7080bba7-a4ad-4f85-bdd7-04c7b8539dc1-tls-certificates podName:7080bba7-a4ad-4f85-bdd7-04c7b8539dc1 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:28.761995509 +0000 UTC m=+298.318739184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/7080bba7-a4ad-4f85-bdd7-04c7b8539dc1-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-p9xw2" (UID: "7080bba7-a4ad-4f85-bdd7-04c7b8539dc1") : secret "prometheus-operator-admission-webhook-tls" not found Jul 22 20:06:28.267385 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.267324 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0458ea69-7f6c-4806-a27c-adaf0d15ad76-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zd8zj\" (UID: \"0458ea69-7f6c-4806-a27c-adaf0d15ad76\") " pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:28.438728 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.438631 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zd8zj" Jul 22 20:06:28.766290 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.766257 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7080bba7-a4ad-4f85-bdd7-04c7b8539dc1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-p9xw2\" (UID: \"7080bba7-a4ad-4f85-bdd7-04c7b8539dc1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" Jul 22 20:06:28.769291 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.769258 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7080bba7-a4ad-4f85-bdd7-04c7b8539dc1-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-p9xw2\" (UID: \"7080bba7-a4ad-4f85-bdd7-04c7b8539dc1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" Jul 22 20:06:28.974759 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:28.974723 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" Jul 22 20:06:29.682143 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:29.682097 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zd8zj"] Jul 22 20:06:29.685394 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:29.685355 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0458ea69_7f6c_4806_a27c_adaf0d15ad76.slice/crio-3656cca6adbdf9b5dd3f7d2e17def8c3eab949de765421e63ca3180694eb4be4 WatchSource:0}: Error finding container 3656cca6adbdf9b5dd3f7d2e17def8c3eab949de765421e63ca3180694eb4be4: Status 404 returned error can't find the container with id 3656cca6adbdf9b5dd3f7d2e17def8c3eab949de765421e63ca3180694eb4be4 Jul 22 20:06:29.702857 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:29.702828 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2"] Jul 22 20:06:29.864457 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:29.864364 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7080bba7_a4ad_4f85_bdd7_04c7b8539dc1.slice/crio-79d3e3671c334e3f4629b718963098ba72f44c21c434665823e84eed184ba6ef WatchSource:0}: Error finding container 79d3e3671c334e3f4629b718963098ba72f44c21c434665823e84eed184ba6ef: Status 404 returned error can't find the container with id 79d3e3671c334e3f4629b718963098ba72f44c21c434665823e84eed184ba6ef Jul 22 20:06:29.899433 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:29.899396 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hqpcf" Jul 22 20:06:29.936845 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:29.936815 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zd8zj" event={"ID":"0458ea69-7f6c-4806-a27c-adaf0d15ad76","Type":"ContainerStarted","Data":"3656cca6adbdf9b5dd3f7d2e17def8c3eab949de765421e63ca3180694eb4be4"} Jul 22 20:06:29.938008 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:29.937986 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" event={"ID":"7080bba7-a4ad-4f85-bdd7-04c7b8539dc1","Type":"ContainerStarted","Data":"79d3e3671c334e3f4629b718963098ba72f44c21c434665823e84eed184ba6ef"} Jul 22 20:06:30.849590 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:30.849513 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:06:30.850084 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:30.850063 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:06:30.877059 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:30.877036 2553 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 20:06:30.944546 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:30.944355 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-848c569bfc-hqbqv" event={"ID":"d9a0989e-c780-4f1e-8de4-25a762111ee7","Type":"ContainerStarted","Data":"122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a"} Jul 22 20:06:30.963734 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:30.963675 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-848c569bfc-hqbqv" podStartSLOduration=2.135376189 podStartE2EDuration="5.963656185s" totalImagesPullingTime="3.828279996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:30.962863799 +0000 UTC m=+300.519607495" watchObservedRunningTime="2026-07-22 20:06:30.963656185 +0000 UTC m=+300.520399879" Jul 22 20:06:31.951352 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:31.951286 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" event={"ID":"7080bba7-a4ad-4f85-bdd7-04c7b8539dc1","Type":"ContainerStarted","Data":"561297f50b20d553539bcd24797e122c18249fd092b6483a807bb2474df1e4a8"} Jul 22 20:06:31.970709 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:31.969921 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" podStartSLOduration=2.4639892310000002 podStartE2EDuration="3.969896937s" totalImagesPullingTime="1.505907706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:31.967510935 +0000 UTC m=+301.524254630" watchObservedRunningTime="2026-07-22 20:06:31.969896937 +0000 UTC m=+301.526640633" Jul 22 20:06:32.955656 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:32.955619 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" Jul 22 20:06:32.961144 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:32.961113 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-p9xw2" Jul 22 20:06:35.913007 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:35.912968 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:35.913515 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:35.913051 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:35.918969 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:35.918946 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:35.971809 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:35.971781 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:06:36.870369 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:36.870336 2553 patch_prober.go:28] interesting pod/image-registry-579b98c4-8z9rg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 22 20:06:36.870561 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:36.870388 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" podUID="da2a3c08-e156-44fb-8571-5c3add922c89" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 20:06:38.000904 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:38.000867 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:06:38.886644 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:38.886612 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:06:40.497396 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.497351 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-6jqxp"] Jul 22 20:06:40.538185 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.536044 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qlg52"] Jul 22 20:06:40.538185 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.536203 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.539475 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.538916 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vbhb8\"" Jul 22 20:06:40.539475 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.538941 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 22 20:06:40.539663 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.539516 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 22 20:06:40.540107 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.539749 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 20:06:40.540107 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.539909 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 20:06:40.540107 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.539913 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 22 20:06:40.540760 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.540597 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 20:06:40.542583 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.542490 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-6jqxp"] Jul 22 20:06:40.542702 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.542684 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.545464 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.545175 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 20:06:40.545464 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.545404 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 20:06:40.545850 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.545687 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-krxkr\"" Jul 22 20:06:40.545850 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.545804 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 20:06:40.678640 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.678580 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/995e6011-ac8e-49f7-91f8-b6c0841f130a-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.678827 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.678670 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98d2q\" (UniqueName: \"kubernetes.io/projected/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-api-access-98d2q\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.678827 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.678729 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-textfile\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.678827 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.678800 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnttq\" (UniqueName: \"kubernetes.io/projected/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-kube-api-access-xnttq\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.678840 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-wtmp\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.678884 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-sys\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.678909 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-root\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.678961 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.679008 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.679100 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-tls\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.679136 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.679190 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.679223 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/995e6011-ac8e-49f7-91f8-b6c0841f130a-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.679256 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.679625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.679280 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-metrics-client-ca\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.780172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780082 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.780172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780139 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.780608 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780584 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-tls\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.780705 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780627 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.780705 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780678 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.780817 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780708 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/995e6011-ac8e-49f7-91f8-b6c0841f130a-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.780817 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780739 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.780817 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780763 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-metrics-client-ca\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.780817 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780799 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/995e6011-ac8e-49f7-91f8-b6c0841f130a-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.781028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780825 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98d2q\" (UniqueName: \"kubernetes.io/projected/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-api-access-98d2q\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.781028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780858 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-textfile\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.781028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780896 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.781028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780917 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnttq\" (UniqueName: \"kubernetes.io/projected/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-kube-api-access-xnttq\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.781028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780967 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-wtmp\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.781028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.780994 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-sys\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.781028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.781022 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-root\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.781346 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.781115 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-root\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.781346 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.781223 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-wtmp\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.781346 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.781265 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-sys\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.782002 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.781708 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-metrics-client-ca\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.782002 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.781914 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/995e6011-ac8e-49f7-91f8-b6c0841f130a-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.782297 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.782273 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/995e6011-ac8e-49f7-91f8-b6c0841f130a-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.782551 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.782530 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-textfile\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.782641 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.782566 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.784077 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.784049 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.784319 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.784293 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-node-exporter-tls\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.785602 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.785578 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.786822 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.786797 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.806251 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.806218 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnttq\" (UniqueName: \"kubernetes.io/projected/11ce996a-40c1-4ee5-a4ea-19f07f6b89d8-kube-api-access-xnttq\") pod \"node-exporter-qlg52\" (UID: \"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8\") " pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:40.807941 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.807916 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98d2q\" (UniqueName: \"kubernetes.io/projected/995e6011-ac8e-49f7-91f8-b6c0841f130a-kube-api-access-98d2q\") pod \"kube-state-metrics-56c5594674-6jqxp\" (UID: \"995e6011-ac8e-49f7-91f8-b6c0841f130a\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.853692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.853654 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" Jul 22 20:06:40.861614 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:40.861588 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qlg52" Jul 22 20:06:41.586174 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.586138 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 20:06:41.590657 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.590630 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.595393 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.595230 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 22 20:06:41.595393 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.595353 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 22 20:06:41.595393 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.595373 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 22 20:06:41.596182 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.595504 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 22 20:06:41.596182 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.595641 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2ztcb\"" Jul 22 20:06:41.596182 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.595778 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 22 20:06:41.596182 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.595819 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 22 20:06:41.596182 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.595836 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 22 20:06:41.596182 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.596069 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 22 20:06:41.597692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.597673 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 22 20:06:41.617550 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.617403 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 20:06:41.658319 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:41.658275 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ce996a_40c1_4ee5_a4ea_19f07f6b89d8.slice/crio-bb3e1f3b375020c937c6c10ee04b703993f0488504a612a66f50c70c722147b4 WatchSource:0}: Error finding container bb3e1f3b375020c937c6c10ee04b703993f0488504a612a66f50c70c722147b4: Status 404 returned error can't find the container with id bb3e1f3b375020c937c6c10ee04b703993f0488504a612a66f50c70c722147b4 Jul 22 20:06:41.660749 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.660721 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 20:06:41.689988 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.689941 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690161 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.689999 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690161 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690121 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-config-out\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690161 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690149 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-config-volume\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690289 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690178 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690289 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690233 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-web-config\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690289 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690278 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690388 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690305 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rnbk\" (UniqueName: \"kubernetes.io/projected/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-kube-api-access-4rnbk\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690388 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690340 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690388 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690368 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690518 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690397 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690518 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690454 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.690518 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.690492 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.791986 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.791933 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-config-out\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.791986 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.791988 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-config-volume\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792226 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792019 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792226 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792051 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-web-config\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792226 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792083 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792226 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792110 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rnbk\" (UniqueName: \"kubernetes.io/projected/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-kube-api-access-4rnbk\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792226 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792156 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792226 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792182 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792226 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792211 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792609 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792248 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792609 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792286 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792609 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792320 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792609 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792349 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.792609 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.792453 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.793477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.793225 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.794511 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:06:41.794259 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-alertmanager-trusted-ca-bundle podName:0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249 nodeName:}" failed. No retries permitted until 2026-07-22 20:06:42.294237657 +0000 UTC m=+311.850981337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249") : configmap references non-existent config key: ca-bundle.crt Jul 22 20:06:41.799472 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.798046 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.805997 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.805772 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-config-volume\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.807305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.806340 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-config-out\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.807305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.806745 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.807305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.806761 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.807305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.806960 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.807305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.807142 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.807305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.807262 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-web-config\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.807830 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.807799 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rnbk\" (UniqueName: \"kubernetes.io/projected/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-kube-api-access-4rnbk\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.808835 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.808815 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:41.811563 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.811534 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-6jqxp"] Jul 22 20:06:41.814032 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:41.814006 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995e6011_ac8e_49f7_91f8_b6c0841f130a.slice/crio-1dcaa67d4d088e78b0d8472e9b2477ea5d013363c3bc1631562c262b7aa0f1c2 WatchSource:0}: Error finding container 1dcaa67d4d088e78b0d8472e9b2477ea5d013363c3bc1631562c262b7aa0f1c2: Status 404 returned error can't find the container with id 1dcaa67d4d088e78b0d8472e9b2477ea5d013363c3bc1631562c262b7aa0f1c2 Jul 22 20:06:41.952987 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.952894 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-579b98c4-8z9rg"] Jul 22 20:06:41.992950 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.992906 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zd8zj" event={"ID":"0458ea69-7f6c-4806-a27c-adaf0d15ad76","Type":"ContainerStarted","Data":"16056d3c19f347c150434d1ad24bd333ee6b32f68c966cdc25cc53285a566535"} Jul 22 20:06:41.994557 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.994516 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlg52" event={"ID":"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8","Type":"ContainerStarted","Data":"bb3e1f3b375020c937c6c10ee04b703993f0488504a612a66f50c70c722147b4"} Jul 22 20:06:41.996731 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.996250 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-xhjxq" event={"ID":"ca4b953d-3730-4b93-b12b-feb323a8b179","Type":"ContainerStarted","Data":"87fdabbe005e17f487024734796078270c1691d3b817d333f6ea13a3cacda085"} Jul 22 20:06:41.996907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.996747 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-xhjxq" Jul 22 20:06:41.998555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:41.998521 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" event={"ID":"995e6011-ac8e-49f7-91f8-b6c0841f130a","Type":"ContainerStarted","Data":"1dcaa67d4d088e78b0d8472e9b2477ea5d013363c3bc1631562c262b7aa0f1c2"} Jul 22 20:06:42.014531 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.014465 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-xhjxq" podStartSLOduration=1.467592525 podStartE2EDuration="23.014446391s" totalImagesPullingTime="21.546853866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:42.013274818 +0000 UTC m=+311.570018535" watchObservedRunningTime="2026-07-22 20:06:42.014446391 +0000 UTC m=+311.571190114" Jul 22 20:06:42.016936 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.016904 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-xhjxq" Jul 22 20:06:42.296887 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.296846 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:42.297945 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.297909 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:42.504332 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.504295 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 20:06:42.625660 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.625598 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7"] Jul 22 20:06:42.631980 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.631951 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.636331 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.634377 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 22 20:06:42.636331 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.634667 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q5fgr\"" Jul 22 20:06:42.636331 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.634855 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 22 20:06:42.636331 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.635210 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 22 20:06:42.636331 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.635480 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 22 20:06:42.636331 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.635744 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 22 20:06:42.636331 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.635950 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dcolu4jfap84o\"" Jul 22 20:06:42.655176 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.644969 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7"] Jul 22 20:06:42.708648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.708541 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 20:06:42.713715 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:42.713663 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0acbcbe8_aeea_4ef9_b1e5_3ca9cef33249.slice/crio-ff20815c1c79f030b932c25975291110b0eb4d2e9d5058be15c06c6d91785636 WatchSource:0}: Error finding container ff20815c1c79f030b932c25975291110b0eb4d2e9d5058be15c06c6d91785636: Status 404 returned error can't find the container with id ff20815c1c79f030b932c25975291110b0eb4d2e9d5058be15c06c6d91785636 Jul 22 20:06:42.803280 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.803230 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.803495 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.803300 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.803495 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.803330 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-grpc-tls\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.803495 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.803394 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.803673 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.803534 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-tls\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.803673 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.803566 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e474344e-cbbe-4ffc-86c3-9783990f3e83-metrics-client-ca\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.803673 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.803626 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.803824 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.803687 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsgnn\" (UniqueName: \"kubernetes.io/projected/e474344e-cbbe-4ffc-86c3-9783990f3e83-kube-api-access-rsgnn\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.905231 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.905145 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsgnn\" (UniqueName: \"kubernetes.io/projected/e474344e-cbbe-4ffc-86c3-9783990f3e83-kube-api-access-rsgnn\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.905231 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.905190 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.905488 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.905267 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.905488 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.905354 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-grpc-tls\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.905608 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.905500 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.905988 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.905955 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-tls\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.906118 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.906007 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e474344e-cbbe-4ffc-86c3-9783990f3e83-metrics-client-ca\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.906118 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.906072 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.907517 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.907217 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e474344e-cbbe-4ffc-86c3-9783990f3e83-metrics-client-ca\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.909158 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.909110 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.909256 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.909175 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.910845 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.910809 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.911193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.911169 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.912148 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.912125 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-grpc-tls\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.912639 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.912596 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e474344e-cbbe-4ffc-86c3-9783990f3e83-secret-thanos-querier-tls\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.915028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.915004 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsgnn\" (UniqueName: \"kubernetes.io/projected/e474344e-cbbe-4ffc-86c3-9783990f3e83-kube-api-access-rsgnn\") pod \"thanos-querier-779d5b5c4b-qkxf7\" (UID: \"e474344e-cbbe-4ffc-86c3-9783990f3e83\") " pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:42.967325 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:42.967081 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:43.007127 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:43.006660 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249","Type":"ContainerStarted","Data":"ff20815c1c79f030b932c25975291110b0eb4d2e9d5058be15c06c6d91785636"} Jul 22 20:06:43.194738 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:43.194648 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7"] Jul 22 20:06:43.772205 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:43.772156 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-848c569bfc-hqbqv"] Jul 22 20:06:43.867633 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:43.867590 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode474344e_cbbe_4ffc_86c3_9783990f3e83.slice/crio-44db607dd260c9776b62bd85a0e8fbf8321efdea629a5e7e7adcc00eb742141e WatchSource:0}: Error finding container 44db607dd260c9776b62bd85a0e8fbf8321efdea629a5e7e7adcc00eb742141e: Status 404 returned error can't find the container with id 44db607dd260c9776b62bd85a0e8fbf8321efdea629a5e7e7adcc00eb742141e Jul 22 20:06:44.015095 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:44.015056 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" event={"ID":"e474344e-cbbe-4ffc-86c3-9783990f3e83","Type":"ContainerStarted","Data":"44db607dd260c9776b62bd85a0e8fbf8321efdea629a5e7e7adcc00eb742141e"} Jul 22 20:06:45.029132 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:45.029095 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" event={"ID":"995e6011-ac8e-49f7-91f8-b6c0841f130a","Type":"ContainerStarted","Data":"a84aed397a3722724ba88183723542d4526fdad40ff469b739bd92975d3afb10"} Jul 22 20:06:46.041678 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.041632 2553 generic.go:356] "Generic (PLEG): container finished" podID="11ce996a-40c1-4ee5-a4ea-19f07f6b89d8" containerID="93f62a1ba5bbaaf28962a9df9894cb7607e17152b0522863599b3ecf29dc49bf" exitCode=0 Jul 22 20:06:46.042193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.041791 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlg52" event={"ID":"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8","Type":"ContainerDied","Data":"93f62a1ba5bbaaf28962a9df9894cb7607e17152b0522863599b3ecf29dc49bf"} Jul 22 20:06:46.048790 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.048004 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" event={"ID":"995e6011-ac8e-49f7-91f8-b6c0841f130a","Type":"ContainerStarted","Data":"e6267d890e12ffd3858c768829eb6b39c0c554f239343739022aa7275e39b696"} Jul 22 20:06:46.048790 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.048075 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" event={"ID":"995e6011-ac8e-49f7-91f8-b6c0841f130a","Type":"ContainerStarted","Data":"52aeebc752a0682a4474e4d5bc2eed5925a4001fd41d512c0aeea575b27a845c"} Jul 22 20:06:46.269306 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.269155 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-6jqxp" podStartSLOduration=3.478582848 podStartE2EDuration="6.269128636s" totalImagesPullingTime="2.790545788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:46.084721304 +0000 UTC m=+315.641465012" watchObservedRunningTime="2026-07-22 20:06:46.269128636 +0000 UTC m=+315.825872332" Jul 22 20:06:46.270720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.270685 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58d9c69685-k2hpb"] Jul 22 20:06:46.288477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.288323 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d9c69685-k2hpb"] Jul 22 20:06:46.288477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.288449 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.299723 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.297633 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 20:06:46.347267 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.346937 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-serving-cert\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.347267 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.347003 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-oauth-serving-cert\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.347267 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.347056 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-config\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.347267 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.347099 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-oauth-config\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.347267 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.347146 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-service-ca\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.347267 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.347244 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-trusted-ca-bundle\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.347809 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.347340 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcdnc\" (UniqueName: \"kubernetes.io/projected/be99e38d-ec9b-435f-a986-c6eb0c4e1996-kube-api-access-tcdnc\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.449555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.448047 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-oauth-config\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.449555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.448121 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-service-ca\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.449555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.448190 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-trusted-ca-bundle\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.449555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.448239 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcdnc\" (UniqueName: \"kubernetes.io/projected/be99e38d-ec9b-435f-a986-c6eb0c4e1996-kube-api-access-tcdnc\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.449555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.448297 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-serving-cert\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.449555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.448340 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-oauth-serving-cert\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.449555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.448372 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-config\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.449555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.449177 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-config\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.455523 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.452399 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-service-ca\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.455523 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.452521 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-oauth-config\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.455523 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.453401 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-trusted-ca-bundle\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.455523 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.453998 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-oauth-serving-cert\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.472738 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.456104 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-serving-cert\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.472738 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.462143 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcdnc\" (UniqueName: \"kubernetes.io/projected/be99e38d-ec9b-435f-a986-c6eb0c4e1996-kube-api-access-tcdnc\") pod \"console-58d9c69685-k2hpb\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.606578 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.606498 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:46.937356 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:46.937270 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d9c69685-k2hpb"] Jul 22 20:06:46.942122 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:46.942087 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe99e38d_ec9b_435f_a986_c6eb0c4e1996.slice/crio-9b48abaeb1e534005ed606736ab82bbbf0a5991d31c3b2d780e191d437039b4f WatchSource:0}: Error finding container 9b48abaeb1e534005ed606736ab82bbbf0a5991d31c3b2d780e191d437039b4f: Status 404 returned error can't find the container with id 9b48abaeb1e534005ed606736ab82bbbf0a5991d31c3b2d780e191d437039b4f Jul 22 20:06:47.058116 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:47.057193 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlg52" event={"ID":"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8","Type":"ContainerStarted","Data":"932e6e97b92ed9e001f1c5e4750444e2c0ea79be9f1dbf5dc5fbd4ef1945e765"} Jul 22 20:06:47.058116 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:47.057261 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlg52" event={"ID":"11ce996a-40c1-4ee5-a4ea-19f07f6b89d8","Type":"ContainerStarted","Data":"2bf56b55d90a15759c4c9c9871911f851e6e2c22e46aa446c204f7b089a7c12c"} Jul 22 20:06:47.060618 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:47.060448 2553 generic.go:356] "Generic (PLEG): container finished" podID="0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249" containerID="efde2e85fbaaef5e81dc99a7c048a3487a2f1584ca87b5c56e2a598ec01e23b1" exitCode=0 Jul 22 20:06:47.060618 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:47.060520 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249","Type":"ContainerDied","Data":"efde2e85fbaaef5e81dc99a7c048a3487a2f1584ca87b5c56e2a598ec01e23b1"} Jul 22 20:06:47.066323 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:47.066070 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9c69685-k2hpb" event={"ID":"be99e38d-ec9b-435f-a986-c6eb0c4e1996","Type":"ContainerStarted","Data":"9b48abaeb1e534005ed606736ab82bbbf0a5991d31c3b2d780e191d437039b4f"} Jul 22 20:06:47.079883 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:47.078754 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qlg52" podStartSLOduration=3.551204265 podStartE2EDuration="7.078733555s" totalImagesPullingTime="3.326725023s" totalInitContainerRuntime="200.804267ms" isStatefulPod=true podCreationTimestamp="2026-07-22 20:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:47.076878206 +0000 UTC m=+316.633621901" watchObservedRunningTime="2026-07-22 20:06:47.078733555 +0000 UTC m=+316.635477251" Jul 22 20:06:48.075344 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:48.075260 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9c69685-k2hpb" event={"ID":"be99e38d-ec9b-435f-a986-c6eb0c4e1996","Type":"ContainerStarted","Data":"049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a"} Jul 22 20:06:48.095657 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:48.095514 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58d9c69685-k2hpb" podStartSLOduration=2.09549392 podStartE2EDuration="2.09549392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:48.093846295 +0000 UTC m=+317.650590012" watchObservedRunningTime="2026-07-22 20:06:48.09549392 +0000 UTC m=+317.652237616" Jul 22 20:06:50.494178 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.492951 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d9c69685-k2hpb"] Jul 22 20:06:50.525708 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.525652 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86497d6484-m9qk6"] Jul 22 20:06:50.613111 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.613058 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86497d6484-m9qk6"] Jul 22 20:06:50.613311 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.613205 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.696841 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.696797 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-oauth-config\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.697056 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.696849 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-trusted-ca-bundle\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.697056 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.696948 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-oauth-serving-cert\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.697056 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.697008 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-config\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.697056 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.697047 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs7q9\" (UniqueName: \"kubernetes.io/projected/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-kube-api-access-xs7q9\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.697269 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.697104 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-service-ca\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.697269 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.697203 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-serving-cert\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.797896 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.797783 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-oauth-config\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.797896 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.797833 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-trusted-ca-bundle\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.797896 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.797873 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-oauth-serving-cert\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.798177 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.797902 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-config\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.798177 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.797941 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs7q9\" (UniqueName: \"kubernetes.io/projected/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-kube-api-access-xs7q9\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.798177 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.798144 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-service-ca\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.798308 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.798262 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-serving-cert\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.798737 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.798704 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-oauth-serving-cert\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.798915 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.798812 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-config\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.798981 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.798919 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-trusted-ca-bundle\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.799038 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.799016 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-service-ca\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.800741 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.800703 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-oauth-config\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.800882 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.800823 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-serving-cert\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.805866 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.805831 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs7q9\" (UniqueName: \"kubernetes.io/projected/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-kube-api-access-xs7q9\") pod \"console-86497d6484-m9qk6\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:50.927750 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:50.927698 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:06:51.436021 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:51.435933 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86497d6484-m9qk6"] Jul 22 20:06:51.565109 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:06:51.565027 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e806a3_95dd_4d5c_b7de_5990d27b9aa7.slice/crio-2380bab266f60edc2d268a6cfb7dd6de8392d6cd1c607f8b661000495ebb5e00 WatchSource:0}: Error finding container 2380bab266f60edc2d268a6cfb7dd6de8392d6cd1c607f8b661000495ebb5e00: Status 404 returned error can't find the container with id 2380bab266f60edc2d268a6cfb7dd6de8392d6cd1c607f8b661000495ebb5e00 Jul 22 20:06:52.098378 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:52.098337 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zd8zj" event={"ID":"0458ea69-7f6c-4806-a27c-adaf0d15ad76","Type":"ContainerStarted","Data":"49f19216c25644ffce26d9cb904d3a1bc721b8272cd4807f83d719b7087610e6"} Jul 22 20:06:52.100643 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:52.100611 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86497d6484-m9qk6" event={"ID":"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7","Type":"ContainerStarted","Data":"e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e"} Jul 22 20:06:52.100754 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:52.100673 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86497d6484-m9qk6" event={"ID":"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7","Type":"ContainerStarted","Data":"2380bab266f60edc2d268a6cfb7dd6de8392d6cd1c607f8b661000495ebb5e00"} Jul 22 20:06:52.103145 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:52.103110 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" event={"ID":"e474344e-cbbe-4ffc-86c3-9783990f3e83","Type":"ContainerStarted","Data":"24709079a289fe5a2d7d599db4cecdda6b113c6f2b126b20bb9377a7ce9870a8"} Jul 22 20:06:52.103224 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:52.103173 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" event={"ID":"e474344e-cbbe-4ffc-86c3-9783990f3e83","Type":"ContainerStarted","Data":"25a7eb04ebbdf6427f153b91d82851844a00b91363f85195ab6ef2e2b0e2241c"} Jul 22 20:06:52.118431 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:52.118358 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86497d6484-m9qk6" podStartSLOduration=2.118338745 podStartE2EDuration="2.118338745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:52.11813988 +0000 UTC m=+321.674883588" watchObservedRunningTime="2026-07-22 20:06:52.118338745 +0000 UTC m=+321.675082440" Jul 22 20:06:53.110893 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:53.110843 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249","Type":"ContainerStarted","Data":"f88400338580ca08510c5f771466c45a37b49afeda63ace3c7023027f6b9a757"} Jul 22 20:06:53.111512 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:53.110942 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249","Type":"ContainerStarted","Data":"8f7a48c725b2e718ff554b36d2f137c06327b77a75179bb5552daf36e17458a9"} Jul 22 20:06:53.111512 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:53.110974 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249","Type":"ContainerStarted","Data":"74c784b67e78db0386c800da1fefedca7655c595d1de0ba8975c08a85b542b28"} Jul 22 20:06:53.111512 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:53.111004 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249","Type":"ContainerStarted","Data":"047a2e31046741c132704a7a72100d63a1ed0e615d89108341f8ba45a2fd6229"} Jul 22 20:06:53.111512 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:53.111038 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249","Type":"ContainerStarted","Data":"07a00ccbe832fbc19c1012651712e1741ddaef72a0b7b904c7cda3412408f537"} Jul 22 20:06:53.113647 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:53.113593 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" event={"ID":"e474344e-cbbe-4ffc-86c3-9783990f3e83","Type":"ContainerStarted","Data":"ed28f148f0720d5a1d6389db27d2afd0068087ccc75c2d9a7d2be9f50add2c60"} Jul 22 20:06:54.128658 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:54.128565 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249","Type":"ContainerStarted","Data":"6c0da5f19b1d749387ad580a64908e1ecba653ce7c27f85912b5db94d9879080"} Jul 22 20:06:54.132338 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:54.132299 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" event={"ID":"e474344e-cbbe-4ffc-86c3-9783990f3e83","Type":"ContainerStarted","Data":"e9a0169dada242da2d2dbabbbaeed51e464bbf983733f344e32a47efb4ec5f1a"} Jul 22 20:06:54.132547 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:54.132384 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" event={"ID":"e474344e-cbbe-4ffc-86c3-9783990f3e83","Type":"ContainerStarted","Data":"af62296a21d16ceaa022279b07c6fe4c8a33464d7173fe3191991a66afdf57af"} Jul 22 20:06:54.132547 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:54.132440 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" event={"ID":"e474344e-cbbe-4ffc-86c3-9783990f3e83","Type":"ContainerStarted","Data":"07c403ff3602acb31140dde514e4596b5db877d01c0c01752aa7765d589e7bf5"} Jul 22 20:06:54.132547 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:54.132526 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:06:54.157483 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:54.157356 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.11640842 podStartE2EDuration="13.157330786s" totalImagesPullingTime="10.010568979s" totalInitContainerRuntime="30.353387ms" isStatefulPod=false podCreationTimestamp="2026-07-22 20:06:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:54.155185387 +0000 UTC m=+323.711929087" watchObservedRunningTime="2026-07-22 20:06:54.157330786 +0000 UTC m=+323.714074483" Jul 22 20:06:54.178731 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:54.178665 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" podStartSLOduration=3.4893178000000002 podStartE2EDuration="12.178647076s" totalImagesPullingTime="8.689329276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:06:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:54.176402889 +0000 UTC m=+323.733146591" watchObservedRunningTime="2026-07-22 20:06:54.178647076 +0000 UTC m=+323.735390771" Jul 22 20:06:56.607423 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:56.607357 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:06:58.153197 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:58.153156 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zd8zj" event={"ID":"0458ea69-7f6c-4806-a27c-adaf0d15ad76","Type":"ContainerStarted","Data":"abc0a6933c0f3ca68785bc9ecc99774b0eb6f610f8db556d5fc3c64ad2e10044"} Jul 22 20:06:58.171385 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:06:58.171334 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zd8zj" podStartSLOduration=34.818599779 podStartE2EDuration="1m2.171316562s" totalImagesPullingTime="27.352716783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:05:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:06:58.169176377 +0000 UTC m=+327.725920092" watchObservedRunningTime="2026-07-22 20:06:58.171316562 +0000 UTC m=+327.728060257" Jul 22 20:07:00.143552 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:00.143519 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-779d5b5c4b-qkxf7" Jul 22 20:07:00.928592 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:00.928493 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:07:00.928592 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:00.928551 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:07:00.933633 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:00.933602 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:07:01.166631 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:01.166598 2553 generic.go:356] "Generic (PLEG): container finished" podID="5f315bbc-8593-4ead-9e7e-06b55dee57c5" containerID="8ef75c4193c329ead671ca81119abce9e1d5bd354f1dfe71e2212bf64675ef09" exitCode=0 Jul 22 20:07:01.167088 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:01.166670 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sfgrv" event={"ID":"5f315bbc-8593-4ead-9e7e-06b55dee57c5","Type":"ContainerDied","Data":"8ef75c4193c329ead671ca81119abce9e1d5bd354f1dfe71e2212bf64675ef09"} Jul 22 20:07:01.167238 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:01.167207 2553 scope.go:117] "RemoveContainer" containerID="8ef75c4193c329ead671ca81119abce9e1d5bd354f1dfe71e2212bf64675ef09" Jul 22 20:07:01.170756 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:01.170735 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:07:02.172458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:02.172398 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sfgrv" event={"ID":"5f315bbc-8593-4ead-9e7e-06b55dee57c5","Type":"ContainerStarted","Data":"7169ad022ca151defe4176f35d48f7a24ce785bf1303a849b8e6ab46e11c687f"} Jul 22 20:07:06.977460 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:06.977398 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" podUID="da2a3c08-e156-44fb-8571-5c3add922c89" containerName="registry" containerID="cri-o://eee036901aa246eaec45891d353e47394fa757f40445fe44ac32c72d453a7a5d" gracePeriod=30 Jul 22 20:07:07.194066 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.194028 2553 generic.go:356] "Generic (PLEG): container finished" podID="da2a3c08-e156-44fb-8571-5c3add922c89" containerID="eee036901aa246eaec45891d353e47394fa757f40445fe44ac32c72d453a7a5d" exitCode=0 Jul 22 20:07:07.194215 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.194104 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" event={"ID":"da2a3c08-e156-44fb-8571-5c3add922c89","Type":"ContainerDied","Data":"eee036901aa246eaec45891d353e47394fa757f40445fe44ac32c72d453a7a5d"} Jul 22 20:07:07.264644 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.264609 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:07:07.352183 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.352142 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-image-registry-private-configuration\") pod \"da2a3c08-e156-44fb-8571-5c3add922c89\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " Jul 22 20:07:07.352352 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.352196 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") pod \"da2a3c08-e156-44fb-8571-5c3add922c89\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " Jul 22 20:07:07.352352 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.352239 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da2a3c08-e156-44fb-8571-5c3add922c89-ca-trust-extracted\") pod \"da2a3c08-e156-44fb-8571-5c3add922c89\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " Jul 22 20:07:07.352352 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.352287 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-trusted-ca\") pod \"da2a3c08-e156-44fb-8571-5c3add922c89\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " Jul 22 20:07:07.352352 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.352322 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klsfj\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-kube-api-access-klsfj\") pod \"da2a3c08-e156-44fb-8571-5c3add922c89\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " Jul 22 20:07:07.352352 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.352344 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-installation-pull-secrets\") pod \"da2a3c08-e156-44fb-8571-5c3add922c89\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " Jul 22 20:07:07.352679 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.352389 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-bound-sa-token\") pod \"da2a3c08-e156-44fb-8571-5c3add922c89\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " Jul 22 20:07:07.352679 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.352521 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-registry-certificates\") pod \"da2a3c08-e156-44fb-8571-5c3add922c89\" (UID: \"da2a3c08-e156-44fb-8571-5c3add922c89\") " Jul 22 20:07:07.352926 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.352863 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "da2a3c08-e156-44fb-8571-5c3add922c89" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:07:07.353186 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.353160 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "da2a3c08-e156-44fb-8571-5c3add922c89" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:07:07.354974 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.354947 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "da2a3c08-e156-44fb-8571-5c3add922c89" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:07:07.355203 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.355175 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "da2a3c08-e156-44fb-8571-5c3add922c89" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:07:07.355311 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.355257 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "da2a3c08-e156-44fb-8571-5c3add922c89" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:07:07.355311 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.355295 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "da2a3c08-e156-44fb-8571-5c3add922c89" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:07:07.355440 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.355389 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-kube-api-access-klsfj" (OuterVolumeSpecName: "kube-api-access-klsfj") pod "da2a3c08-e156-44fb-8571-5c3add922c89" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89"). InnerVolumeSpecName "kube-api-access-klsfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:07:07.361060 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.361035 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da2a3c08-e156-44fb-8571-5c3add922c89-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "da2a3c08-e156-44fb-8571-5c3add922c89" (UID: "da2a3c08-e156-44fb-8571-5c3add922c89"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:07:07.453857 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.453822 2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-registry-certificates\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:07.453857 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.453853 2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-image-registry-private-configuration\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:07.453857 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.453864 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-registry-tls\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:07.454087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.453875 2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da2a3c08-e156-44fb-8571-5c3add922c89-ca-trust-extracted\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:07.454087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.453883 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da2a3c08-e156-44fb-8571-5c3add922c89-trusted-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:07.454087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.453891 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klsfj\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-kube-api-access-klsfj\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:07.454087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.453899 2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da2a3c08-e156-44fb-8571-5c3add922c89-installation-pull-secrets\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:07.454087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:07.453908 2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da2a3c08-e156-44fb-8571-5c3add922c89-bound-sa-token\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:08.199558 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:08.199522 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" event={"ID":"da2a3c08-e156-44fb-8571-5c3add922c89","Type":"ContainerDied","Data":"7e08aa087f1b07e4c8713de075d02ba9b9962907395515309b927b077e1ff5ef"} Jul 22 20:07:08.200195 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:08.199578 2553 scope.go:117] "RemoveContainer" containerID="eee036901aa246eaec45891d353e47394fa757f40445fe44ac32c72d453a7a5d" Jul 22 20:07:08.200195 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:08.199536 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-579b98c4-8z9rg" Jul 22 20:07:08.220384 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:08.220356 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-579b98c4-8z9rg"] Jul 22 20:07:08.224682 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:08.224658 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-579b98c4-8z9rg"] Jul 22 20:07:08.803299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:08.803242 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-848c569bfc-hqbqv" podUID="d9a0989e-c780-4f1e-8de4-25a762111ee7" containerName="console" containerID="cri-o://122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a" gracePeriod=15 Jul 22 20:07:09.007029 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.006995 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da2a3c08-e156-44fb-8571-5c3add922c89" path="/var/lib/kubelet/pods/da2a3c08-e156-44fb-8571-5c3add922c89/volumes" Jul 22 20:07:09.094520 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.094492 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:07:09.168639 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.168613 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-oauth-serving-cert\") pod \"d9a0989e-c780-4f1e-8de4-25a762111ee7\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " Jul 22 20:07:09.168829 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.168676 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-service-ca\") pod \"d9a0989e-c780-4f1e-8de4-25a762111ee7\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " Jul 22 20:07:09.168829 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.168708 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-serving-cert\") pod \"d9a0989e-c780-4f1e-8de4-25a762111ee7\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " Jul 22 20:07:09.168829 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.168810 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-oauth-config\") pod \"d9a0989e-c780-4f1e-8de4-25a762111ee7\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " Jul 22 20:07:09.168994 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.168886 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-config\") pod \"d9a0989e-c780-4f1e-8de4-25a762111ee7\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " Jul 22 20:07:09.168994 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.168929 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pr6r\" (UniqueName: \"kubernetes.io/projected/d9a0989e-c780-4f1e-8de4-25a762111ee7-kube-api-access-5pr6r\") pod \"d9a0989e-c780-4f1e-8de4-25a762111ee7\" (UID: \"d9a0989e-c780-4f1e-8de4-25a762111ee7\") " Jul 22 20:07:09.169153 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.169130 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d9a0989e-c780-4f1e-8de4-25a762111ee7" (UID: "d9a0989e-c780-4f1e-8de4-25a762111ee7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:07:09.169220 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.169199 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-service-ca" (OuterVolumeSpecName: "service-ca") pod "d9a0989e-c780-4f1e-8de4-25a762111ee7" (UID: "d9a0989e-c780-4f1e-8de4-25a762111ee7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:07:09.169284 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.169258 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-config" (OuterVolumeSpecName: "console-config") pod "d9a0989e-c780-4f1e-8de4-25a762111ee7" (UID: "d9a0989e-c780-4f1e-8de4-25a762111ee7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:07:09.171014 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.170985 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d9a0989e-c780-4f1e-8de4-25a762111ee7" (UID: "d9a0989e-c780-4f1e-8de4-25a762111ee7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:07:09.171014 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.171004 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d9a0989e-c780-4f1e-8de4-25a762111ee7" (UID: "d9a0989e-c780-4f1e-8de4-25a762111ee7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:07:09.171148 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.171046 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a0989e-c780-4f1e-8de4-25a762111ee7-kube-api-access-5pr6r" (OuterVolumeSpecName: "kube-api-access-5pr6r") pod "d9a0989e-c780-4f1e-8de4-25a762111ee7" (UID: "d9a0989e-c780-4f1e-8de4-25a762111ee7"). InnerVolumeSpecName "kube-api-access-5pr6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:07:09.204792 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.204758 2553 generic.go:356] "Generic (PLEG): container finished" podID="d9a0989e-c780-4f1e-8de4-25a762111ee7" containerID="122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a" exitCode=0 Jul 22 20:07:09.205181 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.204803 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-848c569bfc-hqbqv" Jul 22 20:07:09.205181 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.204847 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-848c569bfc-hqbqv" event={"ID":"d9a0989e-c780-4f1e-8de4-25a762111ee7","Type":"ContainerDied","Data":"122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a"} Jul 22 20:07:09.205181 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.204896 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-848c569bfc-hqbqv" event={"ID":"d9a0989e-c780-4f1e-8de4-25a762111ee7","Type":"ContainerDied","Data":"ade91d56b70ec487b322c18afa5edbf1602e74aab397d7796b27e29136cee385"} Jul 22 20:07:09.205181 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.204917 2553 scope.go:117] "RemoveContainer" containerID="122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a" Jul 22 20:07:09.213283 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.213269 2553 scope.go:117] "RemoveContainer" containerID="122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a" Jul 22 20:07:09.213591 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:07:09.213572 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a\": container with ID starting with 122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a not found: ID does not exist" containerID="122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a" Jul 22 20:07:09.213657 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.213599 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a"} err="failed to get container status \"122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a\": rpc error: code = NotFound desc = could not find container \"122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a\": container with ID starting with 122700daf4e03b661c7f1114e6419927e445f3df79bc03d0a888b4d8e599702a not found: ID does not exist" Jul 22 20:07:09.224645 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.224621 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-848c569bfc-hqbqv"] Jul 22 20:07:09.228840 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.228821 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-848c569bfc-hqbqv"] Jul 22 20:07:09.269611 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.269577 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-serving-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:09.269611 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.269604 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-oauth-config\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:09.269611 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.269614 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-console-config\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:09.269837 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.269624 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pr6r\" (UniqueName: \"kubernetes.io/projected/d9a0989e-c780-4f1e-8de4-25a762111ee7-kube-api-access-5pr6r\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:09.269837 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.269643 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-oauth-serving-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:09.269837 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:09.269652 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a0989e-c780-4f1e-8de4-25a762111ee7-service-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:11.006028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:11.005996 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a0989e-c780-4f1e-8de4-25a762111ee7" path="/var/lib/kubelet/pods/d9a0989e-c780-4f1e-8de4-25a762111ee7/volumes" Jul 22 20:07:15.523995 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:15.523925 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58d9c69685-k2hpb" podUID="be99e38d-ec9b-435f-a986-c6eb0c4e1996" containerName="console" containerID="cri-o://049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a" gracePeriod=15 Jul 22 20:07:15.847613 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:15.847580 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:07:16.033986 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.033942 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcdnc\" (UniqueName: \"kubernetes.io/projected/be99e38d-ec9b-435f-a986-c6eb0c4e1996-kube-api-access-tcdnc\") pod \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " Jul 22 20:07:16.034175 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.033999 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-trusted-ca-bundle\") pod \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " Jul 22 20:07:16.034175 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034040 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-oauth-config\") pod \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " Jul 22 20:07:16.034175 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034065 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-serving-cert\") pod \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " Jul 22 20:07:16.034175 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034091 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-service-ca\") pod \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " Jul 22 20:07:16.034390 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034280 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-oauth-serving-cert\") pod \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " Jul 22 20:07:16.034390 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034358 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-config\") pod \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\" (UID: \"be99e38d-ec9b-435f-a986-c6eb0c4e1996\") " Jul 22 20:07:16.034794 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034520 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-service-ca" (OuterVolumeSpecName: "service-ca") pod "be99e38d-ec9b-435f-a986-c6eb0c4e1996" (UID: "be99e38d-ec9b-435f-a986-c6eb0c4e1996"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:07:16.034794 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034623 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "be99e38d-ec9b-435f-a986-c6eb0c4e1996" (UID: "be99e38d-ec9b-435f-a986-c6eb0c4e1996"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:07:16.034794 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034640 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "be99e38d-ec9b-435f-a986-c6eb0c4e1996" (UID: "be99e38d-ec9b-435f-a986-c6eb0c4e1996"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:07:16.034794 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034726 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-trusted-ca-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:16.034794 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034744 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-service-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:16.034794 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034758 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-oauth-serving-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:16.034794 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.034767 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-config" (OuterVolumeSpecName: "console-config") pod "be99e38d-ec9b-435f-a986-c6eb0c4e1996" (UID: "be99e38d-ec9b-435f-a986-c6eb0c4e1996"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:07:16.036725 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.036690 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be99e38d-ec9b-435f-a986-c6eb0c4e1996-kube-api-access-tcdnc" (OuterVolumeSpecName: "kube-api-access-tcdnc") pod "be99e38d-ec9b-435f-a986-c6eb0c4e1996" (UID: "be99e38d-ec9b-435f-a986-c6eb0c4e1996"). InnerVolumeSpecName "kube-api-access-tcdnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:07:16.036822 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.036728 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "be99e38d-ec9b-435f-a986-c6eb0c4e1996" (UID: "be99e38d-ec9b-435f-a986-c6eb0c4e1996"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:07:16.036822 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.036695 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "be99e38d-ec9b-435f-a986-c6eb0c4e1996" (UID: "be99e38d-ec9b-435f-a986-c6eb0c4e1996"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:07:16.135246 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.135158 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcdnc\" (UniqueName: \"kubernetes.io/projected/be99e38d-ec9b-435f-a986-c6eb0c4e1996-kube-api-access-tcdnc\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:16.135246 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.135196 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-oauth-config\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:16.135246 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.135208 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-serving-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:16.135246 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.135223 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be99e38d-ec9b-435f-a986-c6eb0c4e1996-console-config\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:07:16.243550 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.243511 2553 generic.go:356] "Generic (PLEG): container finished" podID="be99e38d-ec9b-435f-a986-c6eb0c4e1996" containerID="049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a" exitCode=0 Jul 22 20:07:16.243720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.243559 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9c69685-k2hpb" Jul 22 20:07:16.243720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.243594 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9c69685-k2hpb" event={"ID":"be99e38d-ec9b-435f-a986-c6eb0c4e1996","Type":"ContainerDied","Data":"049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a"} Jul 22 20:07:16.243720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.243650 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9c69685-k2hpb" event={"ID":"be99e38d-ec9b-435f-a986-c6eb0c4e1996","Type":"ContainerDied","Data":"9b48abaeb1e534005ed606736ab82bbbf0a5991d31c3b2d780e191d437039b4f"} Jul 22 20:07:16.243720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.243679 2553 scope.go:117] "RemoveContainer" containerID="049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a" Jul 22 20:07:16.256552 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.256532 2553 scope.go:117] "RemoveContainer" containerID="049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a" Jul 22 20:07:16.256873 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:07:16.256851 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a\": container with ID starting with 049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a not found: ID does not exist" containerID="049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a" Jul 22 20:07:16.256956 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.256880 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a"} err="failed to get container status \"049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a\": rpc error: code = NotFound desc = could not find container \"049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a\": container with ID starting with 049bae97a85ef585af8f4cfea3d957280d7d1e6c0d311154afda0e98b36ff59a not found: ID does not exist" Jul 22 20:07:16.264872 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.264837 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d9c69685-k2hpb"] Jul 22 20:07:16.270496 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:16.270468 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58d9c69685-k2hpb"] Jul 22 20:07:17.011343 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:17.011308 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be99e38d-ec9b-435f-a986-c6eb0c4e1996" path="/var/lib/kubelet/pods/be99e38d-ec9b-435f-a986-c6eb0c4e1996/volumes" Jul 22 20:07:44.988505 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:44.988468 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:07:44.990872 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:44.990846 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5419f3c-d6a5-4530-872d-08505f1c1c74-metrics-certs\") pod \"network-metrics-daemon-4cnsx\" (UID: \"b5419f3c-d6a5-4530-872d-08505f1c1c74\") " pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:07:45.203942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:45.203907 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l85mx\"" Jul 22 20:07:45.212732 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:45.212702 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4cnsx" Jul 22 20:07:45.357656 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:45.357624 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4cnsx"] Jul 22 20:07:45.360373 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:07:45.360340 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5419f3c_d6a5_4530_872d_08505f1c1c74.slice/crio-034420ac5510e2f51400939efe9cb36fc9bc1916d1b25c60eb230d3dca4d29b3 WatchSource:0}: Error finding container 034420ac5510e2f51400939efe9cb36fc9bc1916d1b25c60eb230d3dca4d29b3: Status 404 returned error can't find the container with id 034420ac5510e2f51400939efe9cb36fc9bc1916d1b25c60eb230d3dca4d29b3 Jul 22 20:07:46.361425 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:46.361370 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4cnsx" event={"ID":"b5419f3c-d6a5-4530-872d-08505f1c1c74","Type":"ContainerStarted","Data":"034420ac5510e2f51400939efe9cb36fc9bc1916d1b25c60eb230d3dca4d29b3"} Jul 22 20:07:47.366970 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:47.366934 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4cnsx" event={"ID":"b5419f3c-d6a5-4530-872d-08505f1c1c74","Type":"ContainerStarted","Data":"7dfefc5450c72d0c4a87d9803d832c335232f8afc413a8de85afce57d6a77419"} Jul 22 20:07:47.367471 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:47.366987 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4cnsx" event={"ID":"b5419f3c-d6a5-4530-872d-08505f1c1c74","Type":"ContainerStarted","Data":"c53bc176267b2e52a7e545444f4765fa34f9bcba9f77eae5721002dd90ccbdea"} Jul 22 20:07:47.383935 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:47.383885 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4cnsx" podStartSLOduration=374.257808362 podStartE2EDuration="6m15.383869013s" totalImagesPullingTime="1.126060651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:07:47.381945789 +0000 UTC m=+376.938689484" watchObservedRunningTime="2026-07-22 20:07:47.383869013 +0000 UTC m=+376.940612708" Jul 22 20:07:58.346756 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.346718 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-596db6f779-8mhwf"] Jul 22 20:07:58.347193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.347041 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a0989e-c780-4f1e-8de4-25a762111ee7" containerName="console" Jul 22 20:07:58.347193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.347052 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a0989e-c780-4f1e-8de4-25a762111ee7" containerName="console" Jul 22 20:07:58.347193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.347066 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be99e38d-ec9b-435f-a986-c6eb0c4e1996" containerName="console" Jul 22 20:07:58.347193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.347071 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="be99e38d-ec9b-435f-a986-c6eb0c4e1996" containerName="console" Jul 22 20:07:58.347193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.347080 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da2a3c08-e156-44fb-8571-5c3add922c89" containerName="registry" Jul 22 20:07:58.347193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.347085 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="da2a3c08-e156-44fb-8571-5c3add922c89" containerName="registry" Jul 22 20:07:58.347193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.347142 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="da2a3c08-e156-44fb-8571-5c3add922c89" containerName="registry" Jul 22 20:07:58.347193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.347150 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a0989e-c780-4f1e-8de4-25a762111ee7" containerName="console" Jul 22 20:07:58.347193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.347158 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="be99e38d-ec9b-435f-a986-c6eb0c4e1996" containerName="console" Jul 22 20:07:58.349904 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.349876 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.360096 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.360072 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-596db6f779-8mhwf"] Jul 22 20:07:58.494334 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.494303 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-serving-cert\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.494507 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.494356 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-oauth-config\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.494507 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.494476 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-service-ca\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.494594 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.494528 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-config\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.494594 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.494583 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-trusted-ca-bundle\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.494660 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.494635 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnbs7\" (UniqueName: \"kubernetes.io/projected/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-kube-api-access-vnbs7\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.494660 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.494653 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-oauth-serving-cert\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.595360 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.595317 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-trusted-ca-bundle\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.595360 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.595362 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnbs7\" (UniqueName: \"kubernetes.io/projected/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-kube-api-access-vnbs7\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.595648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.595378 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-oauth-serving-cert\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.595648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.595397 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-serving-cert\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.595648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.595453 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-oauth-config\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.595648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.595494 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-service-ca\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.595648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.595535 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-config\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.596332 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.596307 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-config\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.596332 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.596317 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-service-ca\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.596559 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.596317 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-oauth-serving-cert\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.596559 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.596405 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-trusted-ca-bundle\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.598068 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.598005 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-oauth-config\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.598186 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.598169 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-serving-cert\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.602700 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.602677 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnbs7\" (UniqueName: \"kubernetes.io/projected/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-kube-api-access-vnbs7\") pod \"console-596db6f779-8mhwf\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.660800 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.660750 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:07:58.804158 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:58.804124 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-596db6f779-8mhwf"] Jul 22 20:07:58.808080 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:07:58.808051 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a897897_2dbe_42d7_bd18_0c85f73ee0fc.slice/crio-d67653e11d4d6509c2dfa7987b2b749dc671913f473466cb85b81876477d0269 WatchSource:0}: Error finding container d67653e11d4d6509c2dfa7987b2b749dc671913f473466cb85b81876477d0269: Status 404 returned error can't find the container with id d67653e11d4d6509c2dfa7987b2b749dc671913f473466cb85b81876477d0269 Jul 22 20:07:59.418510 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:59.418471 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-596db6f779-8mhwf" event={"ID":"4a897897-2dbe-42d7-bd18-0c85f73ee0fc","Type":"ContainerStarted","Data":"8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c"} Jul 22 20:07:59.419047 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:59.418531 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-596db6f779-8mhwf" event={"ID":"4a897897-2dbe-42d7-bd18-0c85f73ee0fc","Type":"ContainerStarted","Data":"d67653e11d4d6509c2dfa7987b2b749dc671913f473466cb85b81876477d0269"} Jul 22 20:07:59.434307 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:07:59.434257 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-596db6f779-8mhwf" podStartSLOduration=1.434233566 podStartE2EDuration="1.434233566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:07:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:07:59.433531102 +0000 UTC m=+388.990274794" watchObservedRunningTime="2026-07-22 20:07:59.434233566 +0000 UTC m=+388.990977262" Jul 22 20:08:08.661963 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:08.661858 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:08:08.662892 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:08.662003 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:08:08.669982 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:08.669915 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:08:09.465707 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:09.465675 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:08:09.509844 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:09.509802 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86497d6484-m9qk6"] Jul 22 20:08:34.534826 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.534749 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86497d6484-m9qk6" podUID="d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" containerName="console" containerID="cri-o://e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e" gracePeriod=15 Jul 22 20:08:34.789176 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.789098 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:08:34.924358 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924312 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-trusted-ca-bundle\") pod \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " Jul 22 20:08:34.924358 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924361 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-config\") pod \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " Jul 22 20:08:34.924628 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924388 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs7q9\" (UniqueName: \"kubernetes.io/projected/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-kube-api-access-xs7q9\") pod \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " Jul 22 20:08:34.924628 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924438 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-service-ca\") pod \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " Jul 22 20:08:34.924628 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924463 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-oauth-serving-cert\") pod \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " Jul 22 20:08:34.924628 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924514 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-serving-cert\") pod \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " Jul 22 20:08:34.924628 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924543 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-oauth-config\") pod \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\" (UID: \"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7\") " Jul 22 20:08:34.924873 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924838 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" (UID: "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:08:34.924952 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924924 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-service-ca" (OuterVolumeSpecName: "service-ca") pod "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" (UID: "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:08:34.924952 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924933 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-config" (OuterVolumeSpecName: "console-config") pod "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" (UID: "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:08:34.925066 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.924989 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" (UID: "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:08:34.926744 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.926715 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-kube-api-access-xs7q9" (OuterVolumeSpecName: "kube-api-access-xs7q9") pod "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" (UID: "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7"). InnerVolumeSpecName "kube-api-access-xs7q9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:08:34.926744 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.926727 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" (UID: "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:08:34.926884 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:34.926717 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" (UID: "d4e806a3-95dd-4d5c-b7de-5990d27b9aa7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:08:35.025856 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.025812 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-trusted-ca-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:08:35.025856 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.025848 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-config\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:08:35.025856 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.025859 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs7q9\" (UniqueName: \"kubernetes.io/projected/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-kube-api-access-xs7q9\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:08:35.025856 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.025868 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-service-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:08:35.026152 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.025877 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-oauth-serving-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:08:35.026152 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.025886 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-serving-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:08:35.026152 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.025894 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7-console-oauth-config\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:08:35.571379 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.571335 2553 generic.go:356] "Generic (PLEG): container finished" podID="d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" containerID="e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e" exitCode=0 Jul 22 20:08:35.571379 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.571382 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86497d6484-m9qk6" Jul 22 20:08:35.571950 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.571430 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86497d6484-m9qk6" event={"ID":"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7","Type":"ContainerDied","Data":"e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e"} Jul 22 20:08:35.571950 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.571484 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86497d6484-m9qk6" event={"ID":"d4e806a3-95dd-4d5c-b7de-5990d27b9aa7","Type":"ContainerDied","Data":"2380bab266f60edc2d268a6cfb7dd6de8392d6cd1c607f8b661000495ebb5e00"} Jul 22 20:08:35.571950 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.571505 2553 scope.go:117] "RemoveContainer" containerID="e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e" Jul 22 20:08:35.580802 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.580779 2553 scope.go:117] "RemoveContainer" containerID="e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e" Jul 22 20:08:35.581131 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:08:35.581105 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e\": container with ID starting with e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e not found: ID does not exist" containerID="e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e" Jul 22 20:08:35.581181 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.581145 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e"} err="failed to get container status \"e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e\": rpc error: code = NotFound desc = could not find container \"e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e\": container with ID starting with e0b11e06b1a48f88a556d5719803e7e35bb09620a496ca3d3dfd22e6f3108c7e not found: ID does not exist" Jul 22 20:08:35.587807 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.587772 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86497d6484-m9qk6"] Jul 22 20:08:35.592800 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:35.592764 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86497d6484-m9qk6"] Jul 22 20:08:37.006547 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:37.006499 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" path="/var/lib/kubelet/pods/d4e806a3-95dd-4d5c-b7de-5990d27b9aa7/volumes" Jul 22 20:08:46.152509 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.152463 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4"] Jul 22 20:08:46.153026 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.153002 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" containerName="console" Jul 22 20:08:46.153074 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.153032 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" containerName="console" Jul 22 20:08:46.153148 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.153134 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4e806a3-95dd-4d5c-b7de-5990d27b9aa7" containerName="console" Jul 22 20:08:46.156238 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.156211 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.158953 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.158924 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9ngqm\"" Jul 22 20:08:46.159211 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.159193 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 20:08:46.159655 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.159639 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 20:08:46.169879 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.169838 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4"] Jul 22 20:08:46.325967 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.325902 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.325967 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.325973 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msn8n\" (UniqueName: \"kubernetes.io/projected/ddfaf607-a473-4fba-b112-24a65a9e6b0d-kube-api-access-msn8n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.326253 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.326060 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.427556 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.427400 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.427556 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.427501 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msn8n\" (UniqueName: \"kubernetes.io/projected/ddfaf607-a473-4fba-b112-24a65a9e6b0d-kube-api-access-msn8n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.427556 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.427530 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.427907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.427881 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.427978 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.427889 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.435730 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.435700 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msn8n\" (UniqueName: \"kubernetes.io/projected/ddfaf607-a473-4fba-b112-24a65a9e6b0d-kube-api-access-msn8n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.466733 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.466688 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:08:46.615670 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.615630 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4"] Jul 22 20:08:46.618811 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:08:46.618779 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddfaf607_a473_4fba_b112_24a65a9e6b0d.slice/crio-951f69e42d1a2f043496b355bc79daf68f4c990fd61581bb4de91f17f0f5d613 WatchSource:0}: Error finding container 951f69e42d1a2f043496b355bc79daf68f4c990fd61581bb4de91f17f0f5d613: Status 404 returned error can't find the container with id 951f69e42d1a2f043496b355bc79daf68f4c990fd61581bb4de91f17f0f5d613 Jul 22 20:08:46.625670 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:46.625621 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" event={"ID":"ddfaf607-a473-4fba-b112-24a65a9e6b0d","Type":"ContainerStarted","Data":"951f69e42d1a2f043496b355bc79daf68f4c990fd61581bb4de91f17f0f5d613"} Jul 22 20:08:54.667911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:54.667858 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" event={"ID":"ddfaf607-a473-4fba-b112-24a65a9e6b0d","Type":"ContainerStarted","Data":"284a6c69428b521302196c6eff5a30922ecc84cf4c748d238c08747eee01a7f4"} Jul 22 20:08:55.673069 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:55.673027 2553 generic.go:356] "Generic (PLEG): container finished" podID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerID="284a6c69428b521302196c6eff5a30922ecc84cf4c748d238c08747eee01a7f4" exitCode=0 Jul 22 20:08:55.673492 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:55.673116 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" event={"ID":"ddfaf607-a473-4fba-b112-24a65a9e6b0d","Type":"ContainerDied","Data":"284a6c69428b521302196c6eff5a30922ecc84cf4c748d238c08747eee01a7f4"} Jul 22 20:08:58.689458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:58.689393 2553 generic.go:356] "Generic (PLEG): container finished" podID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerID="92b53b433946aba90616f53c65a45e31a02cb8dfd8d502d72ffc90906af6f267" exitCode=0 Jul 22 20:08:58.690051 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:08:58.689487 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" event={"ID":"ddfaf607-a473-4fba-b112-24a65a9e6b0d","Type":"ContainerDied","Data":"92b53b433946aba90616f53c65a45e31a02cb8dfd8d502d72ffc90906af6f267"} Jul 22 20:09:08.738664 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:08.738623 2553 generic.go:356] "Generic (PLEG): container finished" podID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerID="bdca5ebda5f4041be3bafafc5b919da7c90c4101c4d989913b16c15682e4432b" exitCode=0 Jul 22 20:09:08.739168 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:08.738708 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" event={"ID":"ddfaf607-a473-4fba-b112-24a65a9e6b0d","Type":"ContainerDied","Data":"bdca5ebda5f4041be3bafafc5b919da7c90c4101c4d989913b16c15682e4432b"} Jul 22 20:09:09.876183 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:09.876149 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:09:09.932433 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:09.932365 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msn8n\" (UniqueName: \"kubernetes.io/projected/ddfaf607-a473-4fba-b112-24a65a9e6b0d-kube-api-access-msn8n\") pod \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " Jul 22 20:09:09.932778 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:09.932484 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-util\") pod \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " Jul 22 20:09:09.932778 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:09.932525 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-bundle\") pod \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\" (UID: \"ddfaf607-a473-4fba-b112-24a65a9e6b0d\") " Jul 22 20:09:09.933341 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:09.933299 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-bundle" (OuterVolumeSpecName: "bundle") pod "ddfaf607-a473-4fba-b112-24a65a9e6b0d" (UID: "ddfaf607-a473-4fba-b112-24a65a9e6b0d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:09:09.935322 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:09.935284 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddfaf607-a473-4fba-b112-24a65a9e6b0d-kube-api-access-msn8n" (OuterVolumeSpecName: "kube-api-access-msn8n") pod "ddfaf607-a473-4fba-b112-24a65a9e6b0d" (UID: "ddfaf607-a473-4fba-b112-24a65a9e6b0d"). InnerVolumeSpecName "kube-api-access-msn8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:09:09.938504 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:09.938464 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-util" (OuterVolumeSpecName: "util") pod "ddfaf607-a473-4fba-b112-24a65a9e6b0d" (UID: "ddfaf607-a473-4fba-b112-24a65a9e6b0d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:09:10.033477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:10.033346 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msn8n\" (UniqueName: \"kubernetes.io/projected/ddfaf607-a473-4fba-b112-24a65a9e6b0d-kube-api-access-msn8n\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:09:10.033477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:10.033379 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-util\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:09:10.033477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:10.033390 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddfaf607-a473-4fba-b112-24a65a9e6b0d-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:09:10.750125 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:10.750079 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" event={"ID":"ddfaf607-a473-4fba-b112-24a65a9e6b0d","Type":"ContainerDied","Data":"951f69e42d1a2f043496b355bc79daf68f4c990fd61581bb4de91f17f0f5d613"} Jul 22 20:09:10.750125 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:10.750112 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540x5pr4" Jul 22 20:09:10.750365 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:10.750147 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="951f69e42d1a2f043496b355bc79daf68f4c990fd61581bb4de91f17f0f5d613" Jul 22 20:09:14.406956 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.406918 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v"] Jul 22 20:09:14.407458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.407259 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerName="util" Jul 22 20:09:14.407458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.407270 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerName="util" Jul 22 20:09:14.407458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.407285 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerName="pull" Jul 22 20:09:14.407458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.407290 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerName="pull" Jul 22 20:09:14.407458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.407303 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerName="extract" Jul 22 20:09:14.407458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.407309 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerName="extract" Jul 22 20:09:14.407458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.407362 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddfaf607-a473-4fba-b112-24a65a9e6b0d" containerName="extract" Jul 22 20:09:14.409762 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.409730 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" Jul 22 20:09:14.413398 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.413367 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-kw4cv\"" Jul 22 20:09:14.413598 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.413521 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 22 20:09:14.413688 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.413642 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 22 20:09:14.428224 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.428185 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v"] Jul 22 20:09:14.473271 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.473229 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/694f7f25-5230-4a81-9347-62334cc693e9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dnj8v\" (UID: \"694f7f25-5230-4a81-9347-62334cc693e9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" Jul 22 20:09:14.473271 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.473276 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhc7j\" (UniqueName: \"kubernetes.io/projected/694f7f25-5230-4a81-9347-62334cc693e9-kube-api-access-fhc7j\") pod \"cert-manager-operator-controller-manager-55c444bf86-dnj8v\" (UID: \"694f7f25-5230-4a81-9347-62334cc693e9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" Jul 22 20:09:14.574772 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.574727 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/694f7f25-5230-4a81-9347-62334cc693e9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dnj8v\" (UID: \"694f7f25-5230-4a81-9347-62334cc693e9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" Jul 22 20:09:14.574772 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.574778 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhc7j\" (UniqueName: \"kubernetes.io/projected/694f7f25-5230-4a81-9347-62334cc693e9-kube-api-access-fhc7j\") pod \"cert-manager-operator-controller-manager-55c444bf86-dnj8v\" (UID: \"694f7f25-5230-4a81-9347-62334cc693e9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" Jul 22 20:09:14.575122 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.575100 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/694f7f25-5230-4a81-9347-62334cc693e9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dnj8v\" (UID: \"694f7f25-5230-4a81-9347-62334cc693e9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" Jul 22 20:09:14.586200 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.586168 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhc7j\" (UniqueName: \"kubernetes.io/projected/694f7f25-5230-4a81-9347-62334cc693e9-kube-api-access-fhc7j\") pod \"cert-manager-operator-controller-manager-55c444bf86-dnj8v\" (UID: \"694f7f25-5230-4a81-9347-62334cc693e9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" Jul 22 20:09:14.720370 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.720325 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" Jul 22 20:09:14.888240 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:14.888207 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v"] Jul 22 20:09:14.891557 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:09:14.891510 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694f7f25_5230_4a81_9347_62334cc693e9.slice/crio-9e48a237c5eaee71abd8ca6013021802ab321fb035b48d8ff94627cee9accb71 WatchSource:0}: Error finding container 9e48a237c5eaee71abd8ca6013021802ab321fb035b48d8ff94627cee9accb71: Status 404 returned error can't find the container with id 9e48a237c5eaee71abd8ca6013021802ab321fb035b48d8ff94627cee9accb71 Jul 22 20:09:15.774372 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:15.774328 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" event={"ID":"694f7f25-5230-4a81-9347-62334cc693e9","Type":"ContainerStarted","Data":"9e48a237c5eaee71abd8ca6013021802ab321fb035b48d8ff94627cee9accb71"} Jul 22 20:09:17.787797 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:17.787757 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" event={"ID":"694f7f25-5230-4a81-9347-62334cc693e9","Type":"ContainerStarted","Data":"10c84f6e8c2d8379047c57aad93bfdaca310e11fdd6955a1b189992bea15ea60"} Jul 22 20:09:17.814204 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:17.814141 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dnj8v" podStartSLOduration=1.7098870769999999 podStartE2EDuration="3.814122293s" totalImagesPullingTime="2.104235216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:09:17.81366469 +0000 UTC m=+467.370408384" watchObservedRunningTime="2026-07-22 20:09:17.814122293 +0000 UTC m=+467.370865989" Jul 22 20:09:17.976283 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:17.976237 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sz6gq"] Jul 22 20:09:17.978850 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:17.978822 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:17.981444 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:17.981380 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-j98fb\"" Jul 22 20:09:17.981739 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:17.981718 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 20:09:17.981842 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:17.981788 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 20:09:17.990982 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:17.990947 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sz6gq"] Jul 22 20:09:18.105357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.105263 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc90ad50-955d-4e58-9ec2-07a268e8822c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sz6gq\" (UID: \"dc90ad50-955d-4e58-9ec2-07a268e8822c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:18.105357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.105306 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxjp9\" (UniqueName: \"kubernetes.io/projected/dc90ad50-955d-4e58-9ec2-07a268e8822c-kube-api-access-cxjp9\") pod \"cert-manager-webhook-6486ffd997-sz6gq\" (UID: \"dc90ad50-955d-4e58-9ec2-07a268e8822c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:18.206034 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.205993 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc90ad50-955d-4e58-9ec2-07a268e8822c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sz6gq\" (UID: \"dc90ad50-955d-4e58-9ec2-07a268e8822c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:18.206034 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.206034 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxjp9\" (UniqueName: \"kubernetes.io/projected/dc90ad50-955d-4e58-9ec2-07a268e8822c-kube-api-access-cxjp9\") pod \"cert-manager-webhook-6486ffd997-sz6gq\" (UID: \"dc90ad50-955d-4e58-9ec2-07a268e8822c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:18.242112 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.242066 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxjp9\" (UniqueName: \"kubernetes.io/projected/dc90ad50-955d-4e58-9ec2-07a268e8822c-kube-api-access-cxjp9\") pod \"cert-manager-webhook-6486ffd997-sz6gq\" (UID: \"dc90ad50-955d-4e58-9ec2-07a268e8822c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:18.245290 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.245247 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc90ad50-955d-4e58-9ec2-07a268e8822c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sz6gq\" (UID: \"dc90ad50-955d-4e58-9ec2-07a268e8822c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:18.304175 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.304132 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:18.465189 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.465159 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sz6gq"] Jul 22 20:09:18.467900 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:09:18.467858 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc90ad50_955d_4e58_9ec2_07a268e8822c.slice/crio-41a12d6edb0c8d794f007b038d7c60ed2f2ac7954649b37f7e26bda0f7acc5fd WatchSource:0}: Error finding container 41a12d6edb0c8d794f007b038d7c60ed2f2ac7954649b37f7e26bda0f7acc5fd: Status 404 returned error can't find the container with id 41a12d6edb0c8d794f007b038d7c60ed2f2ac7954649b37f7e26bda0f7acc5fd Jul 22 20:09:18.794268 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.794229 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" event={"ID":"dc90ad50-955d-4e58-9ec2-07a268e8822c","Type":"ContainerStarted","Data":"41a12d6edb0c8d794f007b038d7c60ed2f2ac7954649b37f7e26bda0f7acc5fd"} Jul 22 20:09:18.880946 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.880908 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx"] Jul 22 20:09:18.884153 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.884130 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:18.887974 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.887944 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 20:09:18.888270 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.888254 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 20:09:18.888736 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.888717 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9ngqm\"" Jul 22 20:09:18.894357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:18.894316 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx"] Jul 22 20:09:19.013831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.013791 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.014023 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.013845 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.014023 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.013965 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p58d\" (UniqueName: \"kubernetes.io/projected/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-kube-api-access-2p58d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.115224 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.115122 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.115224 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.115171 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.115224 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.115208 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p58d\" (UniqueName: \"kubernetes.io/projected/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-kube-api-access-2p58d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.115666 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.115643 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.115666 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.115658 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.127031 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.126985 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p58d\" (UniqueName: \"kubernetes.io/projected/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-kube-api-access-2p58d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.196535 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.196465 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:19.350245 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.350176 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx"] Jul 22 20:09:19.352864 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:09:19.352836 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f2d11e_a77b_4e00_be9a_7f488a7c70d2.slice/crio-61879ce82ce301e68dd03fd6abec4a9a242a824f41c83b057e4ca67715e83e1b WatchSource:0}: Error finding container 61879ce82ce301e68dd03fd6abec4a9a242a824f41c83b057e4ca67715e83e1b: Status 404 returned error can't find the container with id 61879ce82ce301e68dd03fd6abec4a9a242a824f41c83b057e4ca67715e83e1b Jul 22 20:09:19.801892 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.801856 2553 generic.go:356] "Generic (PLEG): container finished" podID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerID="dfeeef64318b8fbeebb2c0bd4a30c6ee10709340d1b2446ae8b0bd7ab55a0b94" exitCode=0 Jul 22 20:09:19.802372 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.801944 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" event={"ID":"98f2d11e-a77b-4e00-be9a-7f488a7c70d2","Type":"ContainerDied","Data":"dfeeef64318b8fbeebb2c0bd4a30c6ee10709340d1b2446ae8b0bd7ab55a0b94"} Jul 22 20:09:19.802372 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:19.802003 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" event={"ID":"98f2d11e-a77b-4e00-be9a-7f488a7c70d2","Type":"ContainerStarted","Data":"61879ce82ce301e68dd03fd6abec4a9a242a824f41c83b057e4ca67715e83e1b"} Jul 22 20:09:21.572507 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.572454 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq"] Jul 22 20:09:21.575405 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.575372 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" Jul 22 20:09:21.578437 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.578376 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qjdz2\"" Jul 22 20:09:21.586555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.586475 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq"] Jul 22 20:09:21.638566 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.638523 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4gzc\" (UniqueName: \"kubernetes.io/projected/99b6c306-c20c-4649-bda5-702b56d1fc57-kube-api-access-v4gzc\") pod \"cert-manager-cainjector-559cdd8c87-zwjvq\" (UID: \"99b6c306-c20c-4649-bda5-702b56d1fc57\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" Jul 22 20:09:21.638758 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.638708 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99b6c306-c20c-4649-bda5-702b56d1fc57-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zwjvq\" (UID: \"99b6c306-c20c-4649-bda5-702b56d1fc57\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" Jul 22 20:09:21.740065 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.740021 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99b6c306-c20c-4649-bda5-702b56d1fc57-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zwjvq\" (UID: \"99b6c306-c20c-4649-bda5-702b56d1fc57\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" Jul 22 20:09:21.740289 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.740080 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4gzc\" (UniqueName: \"kubernetes.io/projected/99b6c306-c20c-4649-bda5-702b56d1fc57-kube-api-access-v4gzc\") pod \"cert-manager-cainjector-559cdd8c87-zwjvq\" (UID: \"99b6c306-c20c-4649-bda5-702b56d1fc57\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" Jul 22 20:09:21.748565 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.748523 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99b6c306-c20c-4649-bda5-702b56d1fc57-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zwjvq\" (UID: \"99b6c306-c20c-4649-bda5-702b56d1fc57\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" Jul 22 20:09:21.748565 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.748544 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4gzc\" (UniqueName: \"kubernetes.io/projected/99b6c306-c20c-4649-bda5-702b56d1fc57-kube-api-access-v4gzc\") pod \"cert-manager-cainjector-559cdd8c87-zwjvq\" (UID: \"99b6c306-c20c-4649-bda5-702b56d1fc57\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" Jul 22 20:09:21.814878 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.814838 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" event={"ID":"dc90ad50-955d-4e58-9ec2-07a268e8822c","Type":"ContainerStarted","Data":"c429b07a1c1022ab2a34a7f6aaef7e3e1456f9c64ec43916e6b8c225d98af35f"} Jul 22 20:09:21.815089 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.814992 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:21.832529 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.832460 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" podStartSLOduration=1.911440445 podStartE2EDuration="4.832431114s" totalImagesPullingTime="2.920990669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:09:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:09:21.831583975 +0000 UTC m=+471.388327683" watchObservedRunningTime="2026-07-22 20:09:21.832431114 +0000 UTC m=+471.389174808" Jul 22 20:09:21.889657 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:21.889600 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" Jul 22 20:09:22.056968 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:22.056926 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq"] Jul 22 20:09:22.059928 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:09:22.059888 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99b6c306_c20c_4649_bda5_702b56d1fc57.slice/crio-3e234f546ba723f0ff505a520763f50ef0a84c250d5147c2cc67b9bb5dc98f46 WatchSource:0}: Error finding container 3e234f546ba723f0ff505a520763f50ef0a84c250d5147c2cc67b9bb5dc98f46: Status 404 returned error can't find the container with id 3e234f546ba723f0ff505a520763f50ef0a84c250d5147c2cc67b9bb5dc98f46 Jul 22 20:09:22.824087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:22.824038 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" event={"ID":"99b6c306-c20c-4649-bda5-702b56d1fc57","Type":"ContainerStarted","Data":"d0d233ea6048d294c4d0e67c096500e9a46d5afb02a62be019c6b319797a205d"} Jul 22 20:09:22.824672 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:22.824097 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" event={"ID":"99b6c306-c20c-4649-bda5-702b56d1fc57","Type":"ContainerStarted","Data":"3e234f546ba723f0ff505a520763f50ef0a84c250d5147c2cc67b9bb5dc98f46"} Jul 22 20:09:22.839484 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:22.839392 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zwjvq" podStartSLOduration=1.839371283 podStartE2EDuration="1.839371283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:09:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:09:22.838754899 +0000 UTC m=+472.395498598" watchObservedRunningTime="2026-07-22 20:09:22.839371283 +0000 UTC m=+472.396114978" Jul 22 20:09:23.832062 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:23.832024 2553 generic.go:356] "Generic (PLEG): container finished" podID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerID="30a239af1cf12f6f1fe371fa4cd6e3897a213eaddf75fe9c9946cf2c5dafec90" exitCode=0 Jul 22 20:09:23.832470 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:23.832102 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" event={"ID":"98f2d11e-a77b-4e00-be9a-7f488a7c70d2","Type":"ContainerDied","Data":"30a239af1cf12f6f1fe371fa4cd6e3897a213eaddf75fe9c9946cf2c5dafec90"} Jul 22 20:09:24.839824 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:24.839780 2553 generic.go:356] "Generic (PLEG): container finished" podID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerID="a7c28724650549a399b8b1e7c8986790d73c48f796666f59487e6f469e92aa11" exitCode=0 Jul 22 20:09:24.840267 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:24.839846 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" event={"ID":"98f2d11e-a77b-4e00-be9a-7f488a7c70d2","Type":"ContainerDied","Data":"a7c28724650549a399b8b1e7c8986790d73c48f796666f59487e6f469e92aa11"} Jul 22 20:09:25.980373 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:25.980336 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:26.084440 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.084364 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p58d\" (UniqueName: \"kubernetes.io/projected/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-kube-api-access-2p58d\") pod \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " Jul 22 20:09:26.084695 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.084469 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-bundle\") pod \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " Jul 22 20:09:26.084695 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.084504 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-util\") pod \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\" (UID: \"98f2d11e-a77b-4e00-be9a-7f488a7c70d2\") " Jul 22 20:09:26.085020 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.084952 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-bundle" (OuterVolumeSpecName: "bundle") pod "98f2d11e-a77b-4e00-be9a-7f488a7c70d2" (UID: "98f2d11e-a77b-4e00-be9a-7f488a7c70d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:09:26.087693 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.087651 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-kube-api-access-2p58d" (OuterVolumeSpecName: "kube-api-access-2p58d") pod "98f2d11e-a77b-4e00-be9a-7f488a7c70d2" (UID: "98f2d11e-a77b-4e00-be9a-7f488a7c70d2"). InnerVolumeSpecName "kube-api-access-2p58d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:09:26.090151 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.090108 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-util" (OuterVolumeSpecName: "util") pod "98f2d11e-a77b-4e00-be9a-7f488a7c70d2" (UID: "98f2d11e-a77b-4e00-be9a-7f488a7c70d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:09:26.186174 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.185935 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:09:26.186174 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.186009 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-util\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:09:26.186174 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.186025 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p58d\" (UniqueName: \"kubernetes.io/projected/98f2d11e-a77b-4e00-be9a-7f488a7c70d2-kube-api-access-2p58d\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:09:26.855071 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.855035 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" Jul 22 20:09:26.855242 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.855032 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftm8hx" event={"ID":"98f2d11e-a77b-4e00-be9a-7f488a7c70d2","Type":"ContainerDied","Data":"61879ce82ce301e68dd03fd6abec4a9a242a824f41c83b057e4ca67715e83e1b"} Jul 22 20:09:26.855242 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:26.855162 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61879ce82ce301e68dd03fd6abec4a9a242a824f41c83b057e4ca67715e83e1b" Jul 22 20:09:27.827577 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:27.827538 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sz6gq" Jul 22 20:09:30.635110 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.635068 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jdsbr"] Jul 22 20:09:30.635564 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.635494 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerName="util" Jul 22 20:09:30.635564 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.635508 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerName="util" Jul 22 20:09:30.635564 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.635528 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerName="extract" Jul 22 20:09:30.635564 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.635536 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerName="extract" Jul 22 20:09:30.635564 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.635550 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerName="pull" Jul 22 20:09:30.635564 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.635558 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerName="pull" Jul 22 20:09:30.635757 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.635633 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f2d11e-a77b-4e00-be9a-7f488a7c70d2" containerName="extract" Jul 22 20:09:30.642204 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.642161 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jdsbr" Jul 22 20:09:30.644464 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.644405 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-8hg2v\"" Jul 22 20:09:30.653268 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.650959 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jdsbr"] Jul 22 20:09:30.729203 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.729143 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d23f5548-0896-49f7-9db6-338ddd09d730-bound-sa-token\") pod \"cert-manager-55b4d954d5-jdsbr\" (UID: \"d23f5548-0896-49f7-9db6-338ddd09d730\") " pod="cert-manager/cert-manager-55b4d954d5-jdsbr" Jul 22 20:09:30.729203 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.729204 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg2mk\" (UniqueName: \"kubernetes.io/projected/d23f5548-0896-49f7-9db6-338ddd09d730-kube-api-access-vg2mk\") pod \"cert-manager-55b4d954d5-jdsbr\" (UID: \"d23f5548-0896-49f7-9db6-338ddd09d730\") " pod="cert-manager/cert-manager-55b4d954d5-jdsbr" Jul 22 20:09:30.830397 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.830348 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg2mk\" (UniqueName: \"kubernetes.io/projected/d23f5548-0896-49f7-9db6-338ddd09d730-kube-api-access-vg2mk\") pod \"cert-manager-55b4d954d5-jdsbr\" (UID: \"d23f5548-0896-49f7-9db6-338ddd09d730\") " pod="cert-manager/cert-manager-55b4d954d5-jdsbr" Jul 22 20:09:30.830654 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.830493 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d23f5548-0896-49f7-9db6-338ddd09d730-bound-sa-token\") pod \"cert-manager-55b4d954d5-jdsbr\" (UID: \"d23f5548-0896-49f7-9db6-338ddd09d730\") " pod="cert-manager/cert-manager-55b4d954d5-jdsbr" Jul 22 20:09:30.842012 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.841960 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d23f5548-0896-49f7-9db6-338ddd09d730-bound-sa-token\") pod \"cert-manager-55b4d954d5-jdsbr\" (UID: \"d23f5548-0896-49f7-9db6-338ddd09d730\") " pod="cert-manager/cert-manager-55b4d954d5-jdsbr" Jul 22 20:09:30.842212 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.842072 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg2mk\" (UniqueName: \"kubernetes.io/projected/d23f5548-0896-49f7-9db6-338ddd09d730-kube-api-access-vg2mk\") pod \"cert-manager-55b4d954d5-jdsbr\" (UID: \"d23f5548-0896-49f7-9db6-338ddd09d730\") " pod="cert-manager/cert-manager-55b4d954d5-jdsbr" Jul 22 20:09:30.963623 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.963589 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-8hg2v\"" Jul 22 20:09:30.971607 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:30.971576 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jdsbr" Jul 22 20:09:31.128785 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:31.128743 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jdsbr"] Jul 22 20:09:31.132205 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:09:31.132163 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd23f5548_0896_49f7_9db6_338ddd09d730.slice/crio-71596dbfeefd696f1152f1b0922a56666f99c07be8542b8eaa192606e5f234d6 WatchSource:0}: Error finding container 71596dbfeefd696f1152f1b0922a56666f99c07be8542b8eaa192606e5f234d6: Status 404 returned error can't find the container with id 71596dbfeefd696f1152f1b0922a56666f99c07be8542b8eaa192606e5f234d6 Jul 22 20:09:31.885208 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:31.885147 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jdsbr" event={"ID":"d23f5548-0896-49f7-9db6-338ddd09d730","Type":"ContainerStarted","Data":"d82e99a0aad828d0a83e5e52b353ce5d6d423f4b344d45fc5f718d184b9d564b"} Jul 22 20:09:31.885694 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:31.885221 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jdsbr" event={"ID":"d23f5548-0896-49f7-9db6-338ddd09d730","Type":"ContainerStarted","Data":"71596dbfeefd696f1152f1b0922a56666f99c07be8542b8eaa192606e5f234d6"} Jul 22 20:09:31.900741 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:31.900665 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-jdsbr" podStartSLOduration=1.900642413 podStartE2EDuration="1.900642413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:09:31.90018464 +0000 UTC m=+481.456928346" watchObservedRunningTime="2026-07-22 20:09:31.900642413 +0000 UTC m=+481.457386122" Jul 22 20:09:40.699272 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.699222 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz"] Jul 22 20:09:40.704883 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.704851 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:40.707174 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.707132 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 20:09:40.707174 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.707159 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9ngqm\"" Jul 22 20:09:40.707483 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.707132 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 20:09:40.710953 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.710921 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz"] Jul 22 20:09:40.824389 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.824346 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:40.824583 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.824444 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:40.824583 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.824470 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-855wk\" (UniqueName: \"kubernetes.io/projected/b5f8b979-0699-4f2b-ae46-d22307245bf7-kube-api-access-855wk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:40.925668 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.925605 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:40.925912 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.925705 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:40.925912 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.925731 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-855wk\" (UniqueName: \"kubernetes.io/projected/b5f8b979-0699-4f2b-ae46-d22307245bf7-kube-api-access-855wk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:40.926161 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.926126 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:40.926161 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.926164 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:40.935237 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:40.935188 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-855wk\" (UniqueName: \"kubernetes.io/projected/b5f8b979-0699-4f2b-ae46-d22307245bf7-kube-api-access-855wk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:41.024528 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:41.024481 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:41.179312 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:41.179279 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz"] Jul 22 20:09:41.182546 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:09:41.182509 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f8b979_0699_4f2b_ae46_d22307245bf7.slice/crio-2e5bdc1ca7fb48c4c8701189540451a5498c952df73a610f141620acfd6c279c WatchSource:0}: Error finding container 2e5bdc1ca7fb48c4c8701189540451a5498c952df73a610f141620acfd6c279c: Status 404 returned error can't find the container with id 2e5bdc1ca7fb48c4c8701189540451a5498c952df73a610f141620acfd6c279c Jul 22 20:09:41.940300 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:41.940247 2553 generic.go:356] "Generic (PLEG): container finished" podID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerID="bfa135f987cd97426b143499825103ee06b130c0483f18c6fc62eba9ad21c50e" exitCode=0 Jul 22 20:09:41.940823 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:41.940335 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" event={"ID":"b5f8b979-0699-4f2b-ae46-d22307245bf7","Type":"ContainerDied","Data":"bfa135f987cd97426b143499825103ee06b130c0483f18c6fc62eba9ad21c50e"} Jul 22 20:09:41.940823 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:41.940394 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" event={"ID":"b5f8b979-0699-4f2b-ae46-d22307245bf7","Type":"ContainerStarted","Data":"2e5bdc1ca7fb48c4c8701189540451a5498c952df73a610f141620acfd6c279c"} Jul 22 20:09:42.948323 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:42.948232 2553 generic.go:356] "Generic (PLEG): container finished" podID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerID="072a93f8ce20b8c9d8e9a74d420cc1ffb19ebfcaea565c8ac4e3cea2928bc94c" exitCode=0 Jul 22 20:09:42.948323 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:42.948304 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" event={"ID":"b5f8b979-0699-4f2b-ae46-d22307245bf7","Type":"ContainerDied","Data":"072a93f8ce20b8c9d8e9a74d420cc1ffb19ebfcaea565c8ac4e3cea2928bc94c"} Jul 22 20:09:43.955746 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:43.955697 2553 generic.go:356] "Generic (PLEG): container finished" podID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerID="26af2fa7397b886abcddbd7dffd4bf35a9a49924ee370d180102fdea9f8543ca" exitCode=0 Jul 22 20:09:43.956236 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:43.955761 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" event={"ID":"b5f8b979-0699-4f2b-ae46-d22307245bf7","Type":"ContainerDied","Data":"26af2fa7397b886abcddbd7dffd4bf35a9a49924ee370d180102fdea9f8543ca"} Jul 22 20:09:45.128234 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.128198 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:45.164001 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.163955 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-bundle\") pod \"b5f8b979-0699-4f2b-ae46-d22307245bf7\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " Jul 22 20:09:45.164919 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.164876 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-bundle" (OuterVolumeSpecName: "bundle") pod "b5f8b979-0699-4f2b-ae46-d22307245bf7" (UID: "b5f8b979-0699-4f2b-ae46-d22307245bf7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:09:45.264687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.264582 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-855wk\" (UniqueName: \"kubernetes.io/projected/b5f8b979-0699-4f2b-ae46-d22307245bf7-kube-api-access-855wk\") pod \"b5f8b979-0699-4f2b-ae46-d22307245bf7\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " Jul 22 20:09:45.264687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.264634 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-util\") pod \"b5f8b979-0699-4f2b-ae46-d22307245bf7\" (UID: \"b5f8b979-0699-4f2b-ae46-d22307245bf7\") " Jul 22 20:09:45.264895 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.264863 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:09:45.266971 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.266926 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f8b979-0699-4f2b-ae46-d22307245bf7-kube-api-access-855wk" (OuterVolumeSpecName: "kube-api-access-855wk") pod "b5f8b979-0699-4f2b-ae46-d22307245bf7" (UID: "b5f8b979-0699-4f2b-ae46-d22307245bf7"). InnerVolumeSpecName "kube-api-access-855wk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:09:45.270720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.270675 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-util" (OuterVolumeSpecName: "util") pod "b5f8b979-0699-4f2b-ae46-d22307245bf7" (UID: "b5f8b979-0699-4f2b-ae46-d22307245bf7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:09:45.365766 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.365722 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-855wk\" (UniqueName: \"kubernetes.io/projected/b5f8b979-0699-4f2b-ae46-d22307245bf7-kube-api-access-855wk\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:09:45.365766 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.365759 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5f8b979-0699-4f2b-ae46-d22307245bf7-util\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:09:45.969606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.969550 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" event={"ID":"b5f8b979-0699-4f2b-ae46-d22307245bf7","Type":"ContainerDied","Data":"2e5bdc1ca7fb48c4c8701189540451a5498c952df73a610f141620acfd6c279c"} Jul 22 20:09:45.969606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.969594 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcg8bz" Jul 22 20:09:45.969888 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:45.969627 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e5bdc1ca7fb48c4c8701189540451a5498c952df73a610f141620acfd6c279c" Jul 22 20:09:57.993924 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.993880 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl"] Jul 22 20:09:57.994457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.994251 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerName="util" Jul 22 20:09:57.994457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.994262 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerName="util" Jul 22 20:09:57.994457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.994276 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerName="pull" Jul 22 20:09:57.994457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.994282 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerName="pull" Jul 22 20:09:57.994457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.994288 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerName="extract" Jul 22 20:09:57.994457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.994294 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerName="extract" Jul 22 20:09:57.994457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.994349 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f8b979-0699-4f2b-ae46-d22307245bf7" containerName="extract" Jul 22 20:09:57.996711 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.996686 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:57.999200 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.999164 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 20:09:57.999200 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.999181 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 20:09:57.999498 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.999250 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 20:09:57.999498 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.999316 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zqsx9\"" Jul 22 20:09:57.999498 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:57.999379 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 20:09:58.017612 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.017567 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl"] Jul 22 20:09:58.062337 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.062290 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-manifests\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.062571 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.062404 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.062571 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.062476 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.062571 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.062500 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdp5l\" (UniqueName: \"kubernetes.io/projected/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-kube-api-access-rdp5l\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.062571 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.062537 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-tmp\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.163854 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.163787 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-manifests\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.164114 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.163886 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.164114 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.163912 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.164114 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.163934 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdp5l\" (UniqueName: \"kubernetes.io/projected/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-kube-api-access-rdp5l\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.164114 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.163963 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-tmp\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.164339 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.164278 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-manifests\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.166895 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.166798 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-tmp\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.167519 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.167489 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.167652 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.167489 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-webhook-cert\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.175734 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.175695 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdp5l\" (UniqueName: \"kubernetes.io/projected/f0b1a600-adef-40bf-b36f-e99c5f0d9db5-kube-api-access-rdp5l\") pod \"opendatahub-operator-controller-manager-9fc56f4f-zwzkl\" (UID: \"f0b1a600-adef-40bf-b36f-e99c5f0d9db5\") " pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.308464 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.308286 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:09:58.468553 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:58.468522 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl"] Jul 22 20:09:58.471385 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:09:58.471343 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b1a600_adef_40bf_b36f_e99c5f0d9db5.slice/crio-9e456dc4f504375a1ffe02734aaea5a4c97a27a00a5094c310c5955c8382a814 WatchSource:0}: Error finding container 9e456dc4f504375a1ffe02734aaea5a4c97a27a00a5094c310c5955c8382a814: Status 404 returned error can't find the container with id 9e456dc4f504375a1ffe02734aaea5a4c97a27a00a5094c310c5955c8382a814 Jul 22 20:09:59.040756 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:09:59.040709 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" event={"ID":"f0b1a600-adef-40bf-b36f-e99c5f0d9db5","Type":"ContainerStarted","Data":"9e456dc4f504375a1ffe02734aaea5a4c97a27a00a5094c310c5955c8382a814"} Jul 22 20:10:02.060158 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:02.060115 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" event={"ID":"f0b1a600-adef-40bf-b36f-e99c5f0d9db5","Type":"ContainerStarted","Data":"88880ea1009180616a798cb4afb14db74745d601b13e8e832a7a360cdae8ba75"} Jul 22 20:10:04.073181 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:04.073140 2553 generic.go:356] "Generic (PLEG): container finished" podID="f0b1a600-adef-40bf-b36f-e99c5f0d9db5" containerID="88880ea1009180616a798cb4afb14db74745d601b13e8e832a7a360cdae8ba75" exitCode=0 Jul 22 20:10:04.073629 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:04.073212 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" event={"ID":"f0b1a600-adef-40bf-b36f-e99c5f0d9db5","Type":"ContainerDied","Data":"88880ea1009180616a798cb4afb14db74745d601b13e8e832a7a360cdae8ba75"} Jul 22 20:10:05.081171 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:05.081109 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" event={"ID":"f0b1a600-adef-40bf-b36f-e99c5f0d9db5","Type":"ContainerStarted","Data":"686011ba1b4c239fef129c9f0b8763d7fb9654d0afee1d0dcf22f716fd548428"} Jul 22 20:10:05.081622 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:05.081400 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:10:05.118297 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:05.118229 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" podStartSLOduration=2.6730035599999997 podStartE2EDuration="8.118208784s" totalImagesPullingTime="4.286638546s" totalInitContainerRuntime="1.158566678s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:09:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:10:05.115657312 +0000 UTC m=+514.672401026" watchObservedRunningTime="2026-07-22 20:10:05.118208784 +0000 UTC m=+514.674952479" Jul 22 20:10:16.089360 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:16.089322 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9fc56f4f-zwzkl" Jul 22 20:10:20.048827 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.048787 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7"] Jul 22 20:10:20.059277 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.059242 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.078196 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.078157 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 20:10:20.078196 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.078178 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 20:10:20.078508 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.078162 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 20:10:20.078508 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.078275 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 20:10:20.100798 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.100760 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7"] Jul 22 20:10:20.180549 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.177354 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6h2p\" (UniqueName: \"kubernetes.io/projected/ad8de055-bd52-4ba9-8b83-ac0845649cee-kube-api-access-b6h2p\") pod \"kube-auth-proxy-787b995ffc-vfsp7\" (UID: \"ad8de055-bd52-4ba9-8b83-ac0845649cee\") " pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.180549 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.177478 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad8de055-bd52-4ba9-8b83-ac0845649cee-tls-certs\") pod \"kube-auth-proxy-787b995ffc-vfsp7\" (UID: \"ad8de055-bd52-4ba9-8b83-ac0845649cee\") " pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.180549 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.177526 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad8de055-bd52-4ba9-8b83-ac0845649cee-tmp\") pod \"kube-auth-proxy-787b995ffc-vfsp7\" (UID: \"ad8de055-bd52-4ba9-8b83-ac0845649cee\") " pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.278676 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.278628 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6h2p\" (UniqueName: \"kubernetes.io/projected/ad8de055-bd52-4ba9-8b83-ac0845649cee-kube-api-access-b6h2p\") pod \"kube-auth-proxy-787b995ffc-vfsp7\" (UID: \"ad8de055-bd52-4ba9-8b83-ac0845649cee\") " pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.278931 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.278694 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad8de055-bd52-4ba9-8b83-ac0845649cee-tls-certs\") pod \"kube-auth-proxy-787b995ffc-vfsp7\" (UID: \"ad8de055-bd52-4ba9-8b83-ac0845649cee\") " pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.278931 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.278718 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad8de055-bd52-4ba9-8b83-ac0845649cee-tmp\") pod \"kube-auth-proxy-787b995ffc-vfsp7\" (UID: \"ad8de055-bd52-4ba9-8b83-ac0845649cee\") " pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.281449 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.281399 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad8de055-bd52-4ba9-8b83-ac0845649cee-tmp\") pod \"kube-auth-proxy-787b995ffc-vfsp7\" (UID: \"ad8de055-bd52-4ba9-8b83-ac0845649cee\") " pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.281666 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.281634 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad8de055-bd52-4ba9-8b83-ac0845649cee-tls-certs\") pod \"kube-auth-proxy-787b995ffc-vfsp7\" (UID: \"ad8de055-bd52-4ba9-8b83-ac0845649cee\") " pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.296361 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.296328 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6h2p\" (UniqueName: \"kubernetes.io/projected/ad8de055-bd52-4ba9-8b83-ac0845649cee-kube-api-access-b6h2p\") pod \"kube-auth-proxy-787b995ffc-vfsp7\" (UID: \"ad8de055-bd52-4ba9-8b83-ac0845649cee\") " pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.331913 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.331807 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-857jb"] Jul 22 20:10:20.335024 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.334991 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:20.369970 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.369939 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" Jul 22 20:10:20.370497 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.370465 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 22 20:10:20.370661 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.370615 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ps6bb\"" Jul 22 20:10:20.381126 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.381091 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-857jb"] Jul 22 20:10:20.481137 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.481087 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85233d48-b20f-4981-a4ca-19d228a4e537-cert\") pod \"odh-model-controller-7998c67dc4-857jb\" (UID: \"85233d48-b20f-4981-a4ca-19d228a4e537\") " pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:20.481314 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.481222 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jpqf\" (UniqueName: \"kubernetes.io/projected/85233d48-b20f-4981-a4ca-19d228a4e537-kube-api-access-8jpqf\") pod \"odh-model-controller-7998c67dc4-857jb\" (UID: \"85233d48-b20f-4981-a4ca-19d228a4e537\") " pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:20.581907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.581866 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jpqf\" (UniqueName: \"kubernetes.io/projected/85233d48-b20f-4981-a4ca-19d228a4e537-kube-api-access-8jpqf\") pod \"odh-model-controller-7998c67dc4-857jb\" (UID: \"85233d48-b20f-4981-a4ca-19d228a4e537\") " pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:20.582111 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.582012 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85233d48-b20f-4981-a4ca-19d228a4e537-cert\") pod \"odh-model-controller-7998c67dc4-857jb\" (UID: \"85233d48-b20f-4981-a4ca-19d228a4e537\") " pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:20.582163 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:10:20.582136 2553 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 22 20:10:20.582231 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:10:20.582219 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85233d48-b20f-4981-a4ca-19d228a4e537-cert podName:85233d48-b20f-4981-a4ca-19d228a4e537 nodeName:}" failed. No retries permitted until 2026-07-22 20:10:21.082196547 +0000 UTC m=+530.638940227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/85233d48-b20f-4981-a4ca-19d228a4e537-cert") pod "odh-model-controller-7998c67dc4-857jb" (UID: "85233d48-b20f-4981-a4ca-19d228a4e537") : secret "odh-model-controller-webhook-cert" not found Jul 22 20:10:20.583471 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.583444 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7"] Jul 22 20:10:20.585960 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:10:20.585917 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8de055_bd52_4ba9_8b83_ac0845649cee.slice/crio-f1e512e1b1e439f796486b9a2f77c6c067a91f8c4d70fe44b58c4a08631e264b WatchSource:0}: Error finding container f1e512e1b1e439f796486b9a2f77c6c067a91f8c4d70fe44b58c4a08631e264b: Status 404 returned error can't find the container with id f1e512e1b1e439f796486b9a2f77c6c067a91f8c4d70fe44b58c4a08631e264b Jul 22 20:10:20.611259 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:20.611215 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jpqf\" (UniqueName: \"kubernetes.io/projected/85233d48-b20f-4981-a4ca-19d228a4e537-kube-api-access-8jpqf\") pod \"odh-model-controller-7998c67dc4-857jb\" (UID: \"85233d48-b20f-4981-a4ca-19d228a4e537\") " pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:21.087929 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:21.087876 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85233d48-b20f-4981-a4ca-19d228a4e537-cert\") pod \"odh-model-controller-7998c67dc4-857jb\" (UID: \"85233d48-b20f-4981-a4ca-19d228a4e537\") " pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:21.091758 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:21.091720 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85233d48-b20f-4981-a4ca-19d228a4e537-cert\") pod \"odh-model-controller-7998c67dc4-857jb\" (UID: \"85233d48-b20f-4981-a4ca-19d228a4e537\") " pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:21.167458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:21.167399 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" event={"ID":"ad8de055-bd52-4ba9-8b83-ac0845649cee","Type":"ContainerStarted","Data":"f1e512e1b1e439f796486b9a2f77c6c067a91f8c4d70fe44b58c4a08631e264b"} Jul 22 20:10:21.247057 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:21.247017 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:21.427012 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:21.426851 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-857jb"] Jul 22 20:10:21.430479 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:10:21.430430 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85233d48_b20f_4981_a4ca_19d228a4e537.slice/crio-f88ffa8252021ab002ec1a84530adb295bf1369ae3f211f90b594b94b86891d2 WatchSource:0}: Error finding container f88ffa8252021ab002ec1a84530adb295bf1369ae3f211f90b594b94b86891d2: Status 404 returned error can't find the container with id f88ffa8252021ab002ec1a84530adb295bf1369ae3f211f90b594b94b86891d2 Jul 22 20:10:22.178310 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.178266 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-857jb" event={"ID":"85233d48-b20f-4981-a4ca-19d228a4e537","Type":"ContainerStarted","Data":"f88ffa8252021ab002ec1a84530adb295bf1369ae3f211f90b594b94b86891d2"} Jul 22 20:10:22.326300 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.326237 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dw6vc"] Jul 22 20:10:22.330508 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.330455 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:22.336304 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.335878 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 22 20:10:22.373715 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.372627 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dw6vc"] Jul 22 20:10:22.502047 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.501998 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q85zg\" (UniqueName: \"kubernetes.io/projected/d11451bb-b7fa-4604-887e-8c16f46c73c1-kube-api-access-q85zg\") pod \"model-serving-api-54d89bd5fc-dw6vc\" (UID: \"d11451bb-b7fa-4604-887e-8c16f46c73c1\") " pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:22.502374 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.502133 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d11451bb-b7fa-4604-887e-8c16f46c73c1-tls-certs\") pod \"model-serving-api-54d89bd5fc-dw6vc\" (UID: \"d11451bb-b7fa-4604-887e-8c16f46c73c1\") " pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:22.603151 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.603105 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q85zg\" (UniqueName: \"kubernetes.io/projected/d11451bb-b7fa-4604-887e-8c16f46c73c1-kube-api-access-q85zg\") pod \"model-serving-api-54d89bd5fc-dw6vc\" (UID: \"d11451bb-b7fa-4604-887e-8c16f46c73c1\") " pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:22.603377 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.603228 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d11451bb-b7fa-4604-887e-8c16f46c73c1-tls-certs\") pod \"model-serving-api-54d89bd5fc-dw6vc\" (UID: \"d11451bb-b7fa-4604-887e-8c16f46c73c1\") " pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:22.606844 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.606796 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d11451bb-b7fa-4604-887e-8c16f46c73c1-tls-certs\") pod \"model-serving-api-54d89bd5fc-dw6vc\" (UID: \"d11451bb-b7fa-4604-887e-8c16f46c73c1\") " pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:22.626692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.626654 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q85zg\" (UniqueName: \"kubernetes.io/projected/d11451bb-b7fa-4604-887e-8c16f46c73c1-kube-api-access-q85zg\") pod \"model-serving-api-54d89bd5fc-dw6vc\" (UID: \"d11451bb-b7fa-4604-887e-8c16f46c73c1\") " pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:22.654430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.654374 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:22.923374 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:22.923266 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dw6vc"] Jul 22 20:10:23.601705 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:10:23.601668 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11451bb_b7fa_4604_887e_8c16f46c73c1.slice/crio-c68e4ad6f6131a1b941c9670df64540c8b08afac9a91a8f0564e556f8e72b375 WatchSource:0}: Error finding container c68e4ad6f6131a1b941c9670df64540c8b08afac9a91a8f0564e556f8e72b375: Status 404 returned error can't find the container with id c68e4ad6f6131a1b941c9670df64540c8b08afac9a91a8f0564e556f8e72b375 Jul 22 20:10:24.206162 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.206118 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" event={"ID":"d11451bb-b7fa-4604-887e-8c16f46c73c1","Type":"ContainerStarted","Data":"c68e4ad6f6131a1b941c9670df64540c8b08afac9a91a8f0564e556f8e72b375"} Jul 22 20:10:24.755622 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.755533 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-drxxf"] Jul 22 20:10:24.760029 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.759997 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:24.777756 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.777723 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 20:10:24.778225 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.778199 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 20:10:24.780235 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.780204 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6ztjs\"" Jul 22 20:10:24.780816 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.780787 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 20:10:24.780963 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.780788 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 20:10:24.849003 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.848954 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-drxxf"] Jul 22 20:10:24.924848 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.924809 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlvsn\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-kube-api-access-xlvsn\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:24.925022 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.924862 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-token\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:24.925022 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.924893 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:24.925163 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.925035 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:24.925163 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.925086 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-cacerts\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:24.925163 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.925125 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-local-certs\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:24.925163 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:24.925157 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.025790 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.025747 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-local-certs\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.026435 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.026382 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.026563 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.026481 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlvsn\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-kube-api-access-xlvsn\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.026563 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.026543 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-token\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.026683 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.026579 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.026738 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.026680 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.026738 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.026718 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-cacerts\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.027625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.027592 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.029390 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.029363 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-cacerts\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.029390 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.029377 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-local-certs\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.029605 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.029572 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.030275 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.030252 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.050202 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.050162 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlvsn\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-kube-api-access-xlvsn\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.065406 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.065380 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-token\") pod \"istiod-openshift-gateway-7574459656-drxxf\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.074498 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.074462 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:25.217952 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.217887 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" event={"ID":"ad8de055-bd52-4ba9-8b83-ac0845649cee","Type":"ContainerStarted","Data":"9c0cc37ebcafee5472076c76449354135b2f448dce05fb9b340c56133c61a533"} Jul 22 20:10:25.220368 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.220309 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-857jb" event={"ID":"85233d48-b20f-4981-a4ca-19d228a4e537","Type":"ContainerStarted","Data":"809d5dc59de386e52da88b138bac21800bc58db856d8ea56b561a0854519cc4a"} Jul 22 20:10:25.220570 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.220397 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:25.280723 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.280608 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-787b995ffc-vfsp7" podStartSLOduration=1.936758421 podStartE2EDuration="6.28058922s" totalImagesPullingTime="4.343830799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:10:25.275519228 +0000 UTC m=+534.832262923" watchObservedRunningTime="2026-07-22 20:10:25.28058922 +0000 UTC m=+534.837332916" Jul 22 20:10:25.285858 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:10:25.285822 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4fd7d8f_78f3_4a87_83e6_e80a6c2abfc6.slice/crio-3038fce80569403d29149d2191ebd0520dba8f435951a97a4a7950924b994b6d WatchSource:0}: Error finding container 3038fce80569403d29149d2191ebd0520dba8f435951a97a4a7950924b994b6d: Status 404 returned error can't find the container with id 3038fce80569403d29149d2191ebd0520dba8f435951a97a4a7950924b994b6d Jul 22 20:10:25.291193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.291159 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-drxxf"] Jul 22 20:10:25.338963 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:25.338898 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-857jb" podStartSLOduration=1.794838689 podStartE2EDuration="5.338878038s" totalImagesPullingTime="3.544039349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:10:25.338133211 +0000 UTC m=+534.894876920" watchObservedRunningTime="2026-07-22 20:10:25.338878038 +0000 UTC m=+534.895621732" Jul 22 20:10:26.230047 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:26.229996 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" event={"ID":"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6","Type":"ContainerStarted","Data":"3038fce80569403d29149d2191ebd0520dba8f435951a97a4a7950924b994b6d"} Jul 22 20:10:27.239045 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:27.238998 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" event={"ID":"d11451bb-b7fa-4604-887e-8c16f46c73c1","Type":"ContainerStarted","Data":"5c481e38c093f06cceb117a1da88a2deb9b0c7f0acbfaab87509693d4b329ce4"} Jul 22 20:10:27.239574 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:27.239247 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:27.299706 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:27.299630 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" podStartSLOduration=2.614171247 podStartE2EDuration="5.299604326s" totalImagesPullingTime="2.685433079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:10:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:10:27.295216628 +0000 UTC m=+536.851960317" watchObservedRunningTime="2026-07-22 20:10:27.299604326 +0000 UTC m=+536.856348022" Jul 22 20:10:28.437918 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:28.437872 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 20:10:28.438250 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:28.437955 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 20:10:29.255893 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:29.255852 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" event={"ID":"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6","Type":"ContainerStarted","Data":"456c86df665202b182bcf37abbd8508d69d5dce0c676fc6bc62c0d3ac1b81f43"} Jul 22 20:10:29.256107 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:29.256088 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:29.258321 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:29.258283 2553 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-drxxf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 20:10:29.258517 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:29.258349 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" podUID="a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 20:10:29.338502 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:29.338436 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" podStartSLOduration=2.189363847 podStartE2EDuration="5.338390432s" totalImagesPullingTime="3.149026585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:10:29.331860747 +0000 UTC m=+538.888604555" watchObservedRunningTime="2026-07-22 20:10:29.338390432 +0000 UTC m=+538.895134126" Jul 22 20:10:30.262120 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:30.262084 2553 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-drxxf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 20:10:30.262618 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:30.262150 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" podUID="a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 20:10:33.262143 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:33.262108 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:35.530119 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.530071 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs"] Jul 22 20:10:35.535348 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.535316 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:35.550651 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.550617 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 22 20:10:35.556880 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.556842 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fsz6z\"" Jul 22 20:10:35.577909 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.577862 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw"] Jul 22 20:10:35.582174 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.582140 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs"] Jul 22 20:10:35.582333 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.582258 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:10:35.594237 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.594206 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xmlcc\"" Jul 22 20:10:35.594392 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.594304 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 22 20:10:35.611312 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.611277 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw"] Jul 22 20:10:35.625155 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.625108 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dead0468-003c-4f71-8848-5b79e506e482-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2wrrs\" (UID: \"dead0468-003c-4f71-8848-5b79e506e482\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:35.625155 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.625159 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6zqg\" (UniqueName: \"kubernetes.io/projected/dead0468-003c-4f71-8848-5b79e506e482-kube-api-access-z6zqg\") pod \"llmisvc-controller-manager-777c6bd7d5-2wrrs\" (UID: \"dead0468-003c-4f71-8848-5b79e506e482\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:35.726705 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.726665 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dead0468-003c-4f71-8848-5b79e506e482-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2wrrs\" (UID: \"dead0468-003c-4f71-8848-5b79e506e482\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:35.726705 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.726721 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6zqg\" (UniqueName: \"kubernetes.io/projected/dead0468-003c-4f71-8848-5b79e506e482-kube-api-access-z6zqg\") pod \"llmisvc-controller-manager-777c6bd7d5-2wrrs\" (UID: \"dead0468-003c-4f71-8848-5b79e506e482\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:35.726944 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.726768 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5b35131-90f5-4246-9ba5-9c72cb136c00-cert\") pod \"kserve-controller-manager-d6f4b65b8-vn5kw\" (UID: \"e5b35131-90f5-4246-9ba5-9c72cb136c00\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:10:35.726944 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.726809 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxxs6\" (UniqueName: \"kubernetes.io/projected/e5b35131-90f5-4246-9ba5-9c72cb136c00-kube-api-access-jxxs6\") pod \"kserve-controller-manager-d6f4b65b8-vn5kw\" (UID: \"e5b35131-90f5-4246-9ba5-9c72cb136c00\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:10:35.726944 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:10:35.726916 2553 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 22 20:10:35.727048 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:10:35.726983 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dead0468-003c-4f71-8848-5b79e506e482-cert podName:dead0468-003c-4f71-8848-5b79e506e482 nodeName:}" failed. No retries permitted until 2026-07-22 20:10:36.226967094 +0000 UTC m=+545.783710770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/dead0468-003c-4f71-8848-5b79e506e482-cert") pod "llmisvc-controller-manager-777c6bd7d5-2wrrs" (UID: "dead0468-003c-4f71-8848-5b79e506e482") : secret "llmisvc-webhook-server-cert" not found Jul 22 20:10:35.765884 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.765853 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6zqg\" (UniqueName: \"kubernetes.io/projected/dead0468-003c-4f71-8848-5b79e506e482-kube-api-access-z6zqg\") pod \"llmisvc-controller-manager-777c6bd7d5-2wrrs\" (UID: \"dead0468-003c-4f71-8848-5b79e506e482\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:35.827356 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.827228 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5b35131-90f5-4246-9ba5-9c72cb136c00-cert\") pod \"kserve-controller-manager-d6f4b65b8-vn5kw\" (UID: \"e5b35131-90f5-4246-9ba5-9c72cb136c00\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:10:35.827356 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.827326 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxxs6\" (UniqueName: \"kubernetes.io/projected/e5b35131-90f5-4246-9ba5-9c72cb136c00-kube-api-access-jxxs6\") pod \"kserve-controller-manager-d6f4b65b8-vn5kw\" (UID: \"e5b35131-90f5-4246-9ba5-9c72cb136c00\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:10:35.830703 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.830669 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5b35131-90f5-4246-9ba5-9c72cb136c00-cert\") pod \"kserve-controller-manager-d6f4b65b8-vn5kw\" (UID: \"e5b35131-90f5-4246-9ba5-9c72cb136c00\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:10:35.837845 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.837814 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxxs6\" (UniqueName: \"kubernetes.io/projected/e5b35131-90f5-4246-9ba5-9c72cb136c00-kube-api-access-jxxs6\") pod \"kserve-controller-manager-d6f4b65b8-vn5kw\" (UID: \"e5b35131-90f5-4246-9ba5-9c72cb136c00\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:10:35.894071 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:35.894024 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:10:36.165103 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:36.165005 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw"] Jul 22 20:10:36.169027 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:10:36.168992 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b35131_90f5_4246_9ba5_9c72cb136c00.slice/crio-951eddef698a071c1543bf48d8e309f0955b84f2e87f9f78f6db92f7e75abefa WatchSource:0}: Error finding container 951eddef698a071c1543bf48d8e309f0955b84f2e87f9f78f6db92f7e75abefa: Status 404 returned error can't find the container with id 951eddef698a071c1543bf48d8e309f0955b84f2e87f9f78f6db92f7e75abefa Jul 22 20:10:36.230909 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:36.230867 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dead0468-003c-4f71-8848-5b79e506e482-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2wrrs\" (UID: \"dead0468-003c-4f71-8848-5b79e506e482\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:36.232560 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:36.232528 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-857jb" Jul 22 20:10:36.233670 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:36.233638 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dead0468-003c-4f71-8848-5b79e506e482-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2wrrs\" (UID: \"dead0468-003c-4f71-8848-5b79e506e482\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:36.296137 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:36.296093 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" event={"ID":"e5b35131-90f5-4246-9ba5-9c72cb136c00","Type":"ContainerStarted","Data":"951eddef698a071c1543bf48d8e309f0955b84f2e87f9f78f6db92f7e75abefa"} Jul 22 20:10:36.448094 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:36.447995 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:36.678645 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:36.678606 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs"] Jul 22 20:10:36.680294 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:10:36.680258 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddead0468_003c_4f71_8848_5b79e506e482.slice/crio-f441d79b67fe9eb47946237e9f69c7167228ed798b05b5ba4485a5319aedfb9f WatchSource:0}: Error finding container f441d79b67fe9eb47946237e9f69c7167228ed798b05b5ba4485a5319aedfb9f: Status 404 returned error can't find the container with id f441d79b67fe9eb47946237e9f69c7167228ed798b05b5ba4485a5319aedfb9f Jul 22 20:10:37.305285 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:37.305246 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" event={"ID":"dead0468-003c-4f71-8848-5b79e506e482","Type":"ContainerStarted","Data":"f441d79b67fe9eb47946237e9f69c7167228ed798b05b5ba4485a5319aedfb9f"} Jul 22 20:10:38.254030 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:38.253996 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dw6vc" Jul 22 20:10:38.937223 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:38.937157 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-drxxf"] Jul 22 20:10:38.937467 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:38.937434 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" podUID="a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" containerName="discovery" containerID="cri-o://456c86df665202b182bcf37abbd8508d69d5dce0c676fc6bc62c0d3ac1b81f43" gracePeriod=30 Jul 22 20:10:39.262209 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.261985 2553 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-drxxf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:8080/ready\": dial tcp 10.132.0.37:8080: connect: connection refused" start-of-body= Jul 22 20:10:39.262209 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.262067 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" podUID="a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.37:8080/ready\": dial tcp 10.132.0.37:8080: connect: connection refused" Jul 22 20:10:39.325285 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.325249 2553 generic.go:356] "Generic (PLEG): container finished" podID="a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" containerID="456c86df665202b182bcf37abbd8508d69d5dce0c676fc6bc62c0d3ac1b81f43" exitCode=0 Jul 22 20:10:39.325506 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.325398 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" event={"ID":"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6","Type":"ContainerDied","Data":"456c86df665202b182bcf37abbd8508d69d5dce0c676fc6bc62c0d3ac1b81f43"} Jul 22 20:10:39.950289 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.950256 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:39.969123 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.969088 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlvsn\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-kube-api-access-xlvsn\") pod \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " Jul 22 20:10:39.969123 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.969127 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-dns-cert\") pod \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " Jul 22 20:10:39.969307 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.969171 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-kubeconfig\") pod \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " Jul 22 20:10:39.969307 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.969223 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-ca-configmap\") pod \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " Jul 22 20:10:39.969307 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.969242 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-local-certs\") pod \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " Jul 22 20:10:39.969307 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.969264 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-token\") pod \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " Jul 22 20:10:39.969521 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.969312 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-cacerts\") pod \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\" (UID: \"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6\") " Jul 22 20:10:39.969641 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.969613 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" (UID: "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:10:39.971992 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.971928 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" (UID: "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:10:39.972162 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.972026 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-local-certs" (OuterVolumeSpecName: "local-certs") pod "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" (UID: "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:10:39.972162 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.972044 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" (UID: "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:10:39.972162 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.972027 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" (UID: "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:10:39.972277 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.972244 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-kube-api-access-xlvsn" (OuterVolumeSpecName: "kube-api-access-xlvsn") pod "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" (UID: "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6"). InnerVolumeSpecName "kube-api-access-xlvsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:10:39.972954 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:39.972923 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-cacerts" (OuterVolumeSpecName: "cacerts") pod "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" (UID: "a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:10:40.071030 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.070990 2553 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-ca-configmap\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:10:40.071030 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.071028 2553 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-local-certs\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:10:40.071030 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.071044 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-token\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:10:40.071262 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.071056 2553 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-cacerts\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:10:40.071262 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.071069 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlvsn\" (UniqueName: \"kubernetes.io/projected/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-kube-api-access-xlvsn\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:10:40.071262 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.071083 2553 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-csr-dns-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:10:40.071262 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.071096 2553 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6-istio-kubeconfig\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:10:40.332240 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.332202 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" event={"ID":"dead0468-003c-4f71-8848-5b79e506e482","Type":"ContainerStarted","Data":"405e2422ae04f028539919debbc40926188858d17d91edf81e25842f8e34b784"} Jul 22 20:10:40.332240 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.332265 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:10:40.333747 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.333724 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" Jul 22 20:10:40.333747 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.333736 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-drxxf" event={"ID":"a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6","Type":"ContainerDied","Data":"3038fce80569403d29149d2191ebd0520dba8f435951a97a4a7950924b994b6d"} Jul 22 20:10:40.333926 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.333792 2553 scope.go:117] "RemoveContainer" containerID="456c86df665202b182bcf37abbd8508d69d5dce0c676fc6bc62c0d3ac1b81f43" Jul 22 20:10:40.335672 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.335643 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" event={"ID":"e5b35131-90f5-4246-9ba5-9c72cb136c00","Type":"ContainerStarted","Data":"bb201a990929c147393a80d375e86fb650f52bce2b8609677123391ddcc70a39"} Jul 22 20:10:40.335797 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.335742 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:10:40.379489 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.379406 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" podStartSLOduration=2.074381462 podStartE2EDuration="5.379386551s" totalImagesPullingTime="3.305005089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:10:40.377460672 +0000 UTC m=+549.934204367" watchObservedRunningTime="2026-07-22 20:10:40.379386551 +0000 UTC m=+549.936130281" Jul 22 20:10:40.482118 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.482050 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" podStartSLOduration=2.363499516 podStartE2EDuration="5.482031769s" totalImagesPullingTime="3.118532253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:10:40.476856012 +0000 UTC m=+550.033599707" watchObservedRunningTime="2026-07-22 20:10:40.482031769 +0000 UTC m=+550.038775463" Jul 22 20:10:40.518916 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.518881 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-drxxf"] Jul 22 20:10:40.526192 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:40.526157 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-drxxf"] Jul 22 20:10:41.014402 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:10:41.014325 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" path="/var/lib/kubelet/pods/a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6/volumes" Jul 22 20:11:11.345472 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:11.345441 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2wrrs" Jul 22 20:11:11.348471 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:11.348452 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vn5kw" Jul 22 20:11:20.578301 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.578260 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbl7r"] Jul 22 20:11:20.578734 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.578676 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" containerName="discovery" Jul 22 20:11:20.578734 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.578688 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" containerName="discovery" Jul 22 20:11:20.578862 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.578766 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4fd7d8f-78f3-4a87-83e6-e80a6c2abfc6" containerName="discovery" Jul 22 20:11:20.587944 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.587912 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dbl7r" Jul 22 20:11:20.596973 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.596946 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 20:11:20.597194 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.597159 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-z8tgt\"" Jul 22 20:11:20.597473 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.597456 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 20:11:20.599546 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.599513 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbl7r"] Jul 22 20:11:20.741404 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.741364 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgqf4\" (UniqueName: \"kubernetes.io/projected/d6e3fc44-f682-48e8-bf74-7e8375a719f7-kube-api-access-lgqf4\") pod \"kuadrant-operator-catalog-dbl7r\" (UID: \"d6e3fc44-f682-48e8-bf74-7e8375a719f7\") " pod="kuadrant-system/kuadrant-operator-catalog-dbl7r" Jul 22 20:11:20.842949 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.842849 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqf4\" (UniqueName: \"kubernetes.io/projected/d6e3fc44-f682-48e8-bf74-7e8375a719f7-kube-api-access-lgqf4\") pod \"kuadrant-operator-catalog-dbl7r\" (UID: \"d6e3fc44-f682-48e8-bf74-7e8375a719f7\") " pod="kuadrant-system/kuadrant-operator-catalog-dbl7r" Jul 22 20:11:20.866953 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.866914 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbl7r"] Jul 22 20:11:20.867211 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:11:20.867192 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lgqf4], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-catalog-dbl7r" podUID="d6e3fc44-f682-48e8-bf74-7e8375a719f7" Jul 22 20:11:20.877580 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:20.877551 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgqf4\" (UniqueName: \"kubernetes.io/projected/d6e3fc44-f682-48e8-bf74-7e8375a719f7-kube-api-access-lgqf4\") pod \"kuadrant-operator-catalog-dbl7r\" (UID: \"d6e3fc44-f682-48e8-bf74-7e8375a719f7\") " pod="kuadrant-system/kuadrant-operator-catalog-dbl7r" Jul 22 20:11:21.110541 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.110445 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v8xph"] Jul 22 20:11:21.113147 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.113124 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v8xph" Jul 22 20:11:21.143887 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.143849 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v8xph"] Jul 22 20:11:21.245765 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.245723 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5hj2\" (UniqueName: \"kubernetes.io/projected/8e2dac47-0347-4473-9744-7eeefb4ac48a-kube-api-access-c5hj2\") pod \"kuadrant-operator-catalog-v8xph\" (UID: \"8e2dac47-0347-4473-9744-7eeefb4ac48a\") " pod="kuadrant-system/kuadrant-operator-catalog-v8xph" Jul 22 20:11:21.346615 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.346587 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5hj2\" (UniqueName: \"kubernetes.io/projected/8e2dac47-0347-4473-9744-7eeefb4ac48a-kube-api-access-c5hj2\") pod \"kuadrant-operator-catalog-v8xph\" (UID: \"8e2dac47-0347-4473-9744-7eeefb4ac48a\") " pod="kuadrant-system/kuadrant-operator-catalog-v8xph" Jul 22 20:11:21.358481 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.358445 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5hj2\" (UniqueName: \"kubernetes.io/projected/8e2dac47-0347-4473-9744-7eeefb4ac48a-kube-api-access-c5hj2\") pod \"kuadrant-operator-catalog-v8xph\" (UID: \"8e2dac47-0347-4473-9744-7eeefb4ac48a\") " pod="kuadrant-system/kuadrant-operator-catalog-v8xph" Jul 22 20:11:21.423513 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.423401 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v8xph" Jul 22 20:11:21.548582 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.548545 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dbl7r" Jul 22 20:11:21.556742 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.556709 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dbl7r" Jul 22 20:11:21.581382 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.581345 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v8xph"] Jul 22 20:11:21.584664 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:11:21.584622 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2dac47_0347_4473_9744_7eeefb4ac48a.slice/crio-3434eef4b987b9082ef6625f9514f2f58a0a14ded4d8e17bcf52538a4ad58f95 WatchSource:0}: Error finding container 3434eef4b987b9082ef6625f9514f2f58a0a14ded4d8e17bcf52538a4ad58f95: Status 404 returned error can't find the container with id 3434eef4b987b9082ef6625f9514f2f58a0a14ded4d8e17bcf52538a4ad58f95 Jul 22 20:11:21.649273 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.649231 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgqf4\" (UniqueName: \"kubernetes.io/projected/d6e3fc44-f682-48e8-bf74-7e8375a719f7-kube-api-access-lgqf4\") pod \"d6e3fc44-f682-48e8-bf74-7e8375a719f7\" (UID: \"d6e3fc44-f682-48e8-bf74-7e8375a719f7\") " Jul 22 20:11:21.651395 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.651360 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e3fc44-f682-48e8-bf74-7e8375a719f7-kube-api-access-lgqf4" (OuterVolumeSpecName: "kube-api-access-lgqf4") pod "d6e3fc44-f682-48e8-bf74-7e8375a719f7" (UID: "d6e3fc44-f682-48e8-bf74-7e8375a719f7"). InnerVolumeSpecName "kube-api-access-lgqf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:11:21.750650 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:21.750611 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgqf4\" (UniqueName: \"kubernetes.io/projected/d6e3fc44-f682-48e8-bf74-7e8375a719f7-kube-api-access-lgqf4\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:22.554558 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:22.554522 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v8xph" event={"ID":"8e2dac47-0347-4473-9744-7eeefb4ac48a","Type":"ContainerStarted","Data":"3434eef4b987b9082ef6625f9514f2f58a0a14ded4d8e17bcf52538a4ad58f95"} Jul 22 20:11:22.554737 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:22.554603 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dbl7r" Jul 22 20:11:22.609531 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:22.609495 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbl7r"] Jul 22 20:11:22.617552 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:22.617526 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dbl7r"] Jul 22 20:11:23.007421 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:23.007370 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e3fc44-f682-48e8-bf74-7e8375a719f7" path="/var/lib/kubelet/pods/d6e3fc44-f682-48e8-bf74-7e8375a719f7/volumes" Jul 22 20:11:24.567351 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:24.567313 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v8xph" event={"ID":"8e2dac47-0347-4473-9744-7eeefb4ac48a","Type":"ContainerStarted","Data":"df8da70f3414ceb870ceee4b40b4ced291c1dee5c0f2cfcf8daea3b15175b93e"} Jul 22 20:11:24.615205 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:24.615086 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-v8xph" podStartSLOduration=0.842324766 podStartE2EDuration="3.615068339s" totalImagesPullingTime="2.772743573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:11:24.613686761 +0000 UTC m=+594.170430456" watchObservedRunningTime="2026-07-22 20:11:24.615068339 +0000 UTC m=+594.171812035" Jul 22 20:11:30.907404 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:30.907371 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:11:30.908933 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:30.908910 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:11:31.424108 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:31.424071 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-v8xph" Jul 22 20:11:31.426185 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:31.424133 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-v8xph" Jul 22 20:11:31.447225 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:31.447197 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-v8xph" Jul 22 20:11:31.623625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:31.623594 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-v8xph" Jul 22 20:11:36.131437 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.131373 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25"] Jul 22 20:11:36.135238 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.135218 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.138698 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.138679 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x4sxh\"" Jul 22 20:11:36.162297 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.162260 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25"] Jul 22 20:11:36.170227 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.170194 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.170385 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.170240 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6j67\" (UniqueName: \"kubernetes.io/projected/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-kube-api-access-q6j67\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.170385 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.170297 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.271298 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.271246 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.271561 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.271311 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6j67\" (UniqueName: \"kubernetes.io/projected/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-kube-api-access-q6j67\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.271561 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.271345 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.271781 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.271755 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.271848 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.271766 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.288095 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.288064 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6j67\" (UniqueName: \"kubernetes.io/projected/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-kube-api-access-q6j67\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.445136 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.445052 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:36.631535 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.631461 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25"] Jul 22 20:11:36.634672 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:11:36.634647 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6d69b8_8d18_4ad1_9fa0_c82bf8604500.slice/crio-ea690fc91ec50986a6faeb8ee8ab7de994ef5ef5437028130492ffb0d0dd44e8 WatchSource:0}: Error finding container ea690fc91ec50986a6faeb8ee8ab7de994ef5ef5437028130492ffb0d0dd44e8: Status 404 returned error can't find the container with id ea690fc91ec50986a6faeb8ee8ab7de994ef5ef5437028130492ffb0d0dd44e8 Jul 22 20:11:36.775846 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.775814 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs"] Jul 22 20:11:36.779550 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.779529 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:36.794642 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.794601 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs"] Jul 22 20:11:36.877204 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.877167 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:36.877389 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.877262 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:36.877389 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.877311 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78bnc\" (UniqueName: \"kubernetes.io/projected/fdceed12-3f3c-4c8b-ad29-03c60e25e034-kube-api-access-78bnc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:36.978295 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.978263 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:36.978511 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.978316 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78bnc\" (UniqueName: \"kubernetes.io/projected/fdceed12-3f3c-4c8b-ad29-03c60e25e034-kube-api-access-78bnc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:36.978511 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.978346 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:36.978721 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.978697 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:36.978787 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.978717 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:36.991610 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:36.991581 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78bnc\" (UniqueName: \"kubernetes.io/projected/fdceed12-3f3c-4c8b-ad29-03c60e25e034-kube-api-access-78bnc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:37.126685 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.126645 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:37.276469 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.276437 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs"] Jul 22 20:11:37.278321 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:11:37.278292 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdceed12_3f3c_4c8b_ad29_03c60e25e034.slice/crio-66abbad674bd908235431636ba02757d53b6115567610fa2ea0b20474154cb23 WatchSource:0}: Error finding container 66abbad674bd908235431636ba02757d53b6115567610fa2ea0b20474154cb23: Status 404 returned error can't find the container with id 66abbad674bd908235431636ba02757d53b6115567610fa2ea0b20474154cb23 Jul 22 20:11:37.339998 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.339968 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz"] Jul 22 20:11:37.344935 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.344911 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.353295 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.353266 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz"] Jul 22 20:11:37.381889 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.381841 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.382093 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.381899 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.382093 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.381918 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-kube-api-access-j6c8b\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.482391 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.482356 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.482601 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.482451 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.482601 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.482475 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-kube-api-access-j6c8b\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.482766 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.482746 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.482837 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.482819 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.491907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.491885 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-kube-api-access-j6c8b\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.642332 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.642235 2553 generic.go:356] "Generic (PLEG): container finished" podID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerID="816114c8dad3877351db1f2b30247905c9050b74714146aeffd5173d51331c10" exitCode=0 Jul 22 20:11:37.642332 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.642305 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" event={"ID":"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500","Type":"ContainerDied","Data":"816114c8dad3877351db1f2b30247905c9050b74714146aeffd5173d51331c10"} Jul 22 20:11:37.642571 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.642363 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" event={"ID":"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500","Type":"ContainerStarted","Data":"ea690fc91ec50986a6faeb8ee8ab7de994ef5ef5437028130492ffb0d0dd44e8"} Jul 22 20:11:37.644089 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.644064 2553 generic.go:356] "Generic (PLEG): container finished" podID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerID="bbaf7bda0bc0ead698a9d61ff6e8c15744fa214482537bcf330a2362b5fef6fb" exitCode=0 Jul 22 20:11:37.644193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.644118 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" event={"ID":"fdceed12-3f3c-4c8b-ad29-03c60e25e034","Type":"ContainerDied","Data":"bbaf7bda0bc0ead698a9d61ff6e8c15744fa214482537bcf330a2362b5fef6fb"} Jul 22 20:11:37.644193 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.644163 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" event={"ID":"fdceed12-3f3c-4c8b-ad29-03c60e25e034","Type":"ContainerStarted","Data":"66abbad674bd908235431636ba02757d53b6115567610fa2ea0b20474154cb23"} Jul 22 20:11:37.657073 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.657052 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:37.801096 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.800979 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz"] Jul 22 20:11:37.803822 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:11:37.803796 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa322f41_c382_4b42_9a9a_e2e9258aa5b0.slice/crio-cc1b42effc63c5e01733f9dab512548fcdd02cf49e31e65721405333e424c2cc WatchSource:0}: Error finding container cc1b42effc63c5e01733f9dab512548fcdd02cf49e31e65721405333e424c2cc: Status 404 returned error can't find the container with id cc1b42effc63c5e01733f9dab512548fcdd02cf49e31e65721405333e424c2cc Jul 22 20:11:37.953672 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.953627 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n"] Jul 22 20:11:37.961668 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.961633 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:37.968216 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.968190 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n"] Jul 22 20:11:37.987618 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.987589 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:37.987761 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.987628 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvlqn\" (UniqueName: \"kubernetes.io/projected/2094f4b5-883b-4788-ab66-e971a541002a-kube-api-access-tvlqn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:37.987761 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:37.987695 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:38.088161 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.088123 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:38.088161 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.088165 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvlqn\" (UniqueName: \"kubernetes.io/projected/2094f4b5-883b-4788-ab66-e971a541002a-kube-api-access-tvlqn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:38.088397 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.088294 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:38.088568 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.088551 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:38.088669 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.088651 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:38.097955 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.097929 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvlqn\" (UniqueName: \"kubernetes.io/projected/2094f4b5-883b-4788-ab66-e971a541002a-kube-api-access-tvlqn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:38.279591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.279553 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:38.442084 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.442054 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n"] Jul 22 20:11:38.489348 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:11:38.489316 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2094f4b5_883b_4788_ab66_e971a541002a.slice/crio-14375f4bf6fcf6b3b614dd599926da32b02a508cd8e7b7baec96bf8e80663a6f WatchSource:0}: Error finding container 14375f4bf6fcf6b3b614dd599926da32b02a508cd8e7b7baec96bf8e80663a6f: Status 404 returned error can't find the container with id 14375f4bf6fcf6b3b614dd599926da32b02a508cd8e7b7baec96bf8e80663a6f Jul 22 20:11:38.655087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.655046 2553 generic.go:356] "Generic (PLEG): container finished" podID="2094f4b5-883b-4788-ab66-e971a541002a" containerID="0f8be3f9df00ac40e1538d7bc7500902d3983c4b659a9b1cb1f9d5f85e480064" exitCode=0 Jul 22 20:11:38.655279 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.655129 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" event={"ID":"2094f4b5-883b-4788-ab66-e971a541002a","Type":"ContainerDied","Data":"0f8be3f9df00ac40e1538d7bc7500902d3983c4b659a9b1cb1f9d5f85e480064"} Jul 22 20:11:38.655279 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.655191 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" event={"ID":"2094f4b5-883b-4788-ab66-e971a541002a","Type":"ContainerStarted","Data":"14375f4bf6fcf6b3b614dd599926da32b02a508cd8e7b7baec96bf8e80663a6f"} Jul 22 20:11:38.657338 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.657307 2553 generic.go:356] "Generic (PLEG): container finished" podID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerID="f92a3043e3dfa50a448974ab615c2dae72387ee80d573fe4e1fe92a21d0ded4a" exitCode=0 Jul 22 20:11:38.657501 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.657386 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" event={"ID":"aa322f41-c382-4b42-9a9a-e2e9258aa5b0","Type":"ContainerDied","Data":"f92a3043e3dfa50a448974ab615c2dae72387ee80d573fe4e1fe92a21d0ded4a"} Jul 22 20:11:38.657501 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.657455 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" event={"ID":"aa322f41-c382-4b42-9a9a-e2e9258aa5b0","Type":"ContainerStarted","Data":"cc1b42effc63c5e01733f9dab512548fcdd02cf49e31e65721405333e424c2cc"} Jul 22 20:11:38.659608 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.659585 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" event={"ID":"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500","Type":"ContainerStarted","Data":"5296c4f78ff23d677f6c7e9f0e56b3020f299d41bcf59eebfb489aba86468234"} Jul 22 20:11:38.662457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.662428 2553 generic.go:356] "Generic (PLEG): container finished" podID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerID="f30ae7a1f25329c910ece213bb858d470f298e442c42e25aec3d88cea82df210" exitCode=0 Jul 22 20:11:38.662577 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:38.662505 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" event={"ID":"fdceed12-3f3c-4c8b-ad29-03c60e25e034","Type":"ContainerDied","Data":"f30ae7a1f25329c910ece213bb858d470f298e442c42e25aec3d88cea82df210"} Jul 22 20:11:39.670064 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:39.670029 2553 generic.go:356] "Generic (PLEG): container finished" podID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerID="5296c4f78ff23d677f6c7e9f0e56b3020f299d41bcf59eebfb489aba86468234" exitCode=0 Jul 22 20:11:39.670532 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:39.670120 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" event={"ID":"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500","Type":"ContainerDied","Data":"5296c4f78ff23d677f6c7e9f0e56b3020f299d41bcf59eebfb489aba86468234"} Jul 22 20:11:39.672315 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:39.672291 2553 generic.go:356] "Generic (PLEG): container finished" podID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerID="912ffe1beb8ed7d5fb225859521d5a544d7923a97d794e2601ac7685150c9456" exitCode=0 Jul 22 20:11:39.672404 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:39.672373 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" event={"ID":"fdceed12-3f3c-4c8b-ad29-03c60e25e034","Type":"ContainerDied","Data":"912ffe1beb8ed7d5fb225859521d5a544d7923a97d794e2601ac7685150c9456"} Jul 22 20:11:39.674288 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:39.674266 2553 generic.go:356] "Generic (PLEG): container finished" podID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerID="e44375ca5cd5bd5bd3802c7050ecd49356583372b4431dfe9e425a18e252c7ee" exitCode=0 Jul 22 20:11:39.674368 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:39.674309 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" event={"ID":"aa322f41-c382-4b42-9a9a-e2e9258aa5b0","Type":"ContainerDied","Data":"e44375ca5cd5bd5bd3802c7050ecd49356583372b4431dfe9e425a18e252c7ee"} Jul 22 20:11:40.681871 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.681778 2553 generic.go:356] "Generic (PLEG): container finished" podID="2094f4b5-883b-4788-ab66-e971a541002a" containerID="29e2a43792faa4ed51f96696ab9b6b349079c750ef49c834fe7ac68ebfdaadb6" exitCode=0 Jul 22 20:11:40.681871 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.681856 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" event={"ID":"2094f4b5-883b-4788-ab66-e971a541002a","Type":"ContainerDied","Data":"29e2a43792faa4ed51f96696ab9b6b349079c750ef49c834fe7ac68ebfdaadb6"} Jul 22 20:11:40.684092 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.684071 2553 generic.go:356] "Generic (PLEG): container finished" podID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerID="64bf0c493c71b901b0e9e766b265ba35747fca4b2432e24a8132aee479a74bdb" exitCode=0 Jul 22 20:11:40.684187 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.684159 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" event={"ID":"aa322f41-c382-4b42-9a9a-e2e9258aa5b0","Type":"ContainerDied","Data":"64bf0c493c71b901b0e9e766b265ba35747fca4b2432e24a8132aee479a74bdb"} Jul 22 20:11:40.686232 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.686207 2553 generic.go:356] "Generic (PLEG): container finished" podID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerID="0d38ad17f38525fd7df56690525871333b068fa0dccf4889065f8568d75b821d" exitCode=0 Jul 22 20:11:40.686321 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.686244 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" event={"ID":"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500","Type":"ContainerDied","Data":"0d38ad17f38525fd7df56690525871333b068fa0dccf4889065f8568d75b821d"} Jul 22 20:11:40.830683 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.830650 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:40.913453 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.913395 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-util\") pod \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " Jul 22 20:11:40.913613 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.913520 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78bnc\" (UniqueName: \"kubernetes.io/projected/fdceed12-3f3c-4c8b-ad29-03c60e25e034-kube-api-access-78bnc\") pod \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " Jul 22 20:11:40.913613 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.913566 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-bundle\") pod \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\" (UID: \"fdceed12-3f3c-4c8b-ad29-03c60e25e034\") " Jul 22 20:11:40.914296 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.914260 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-bundle" (OuterVolumeSpecName: "bundle") pod "fdceed12-3f3c-4c8b-ad29-03c60e25e034" (UID: "fdceed12-3f3c-4c8b-ad29-03c60e25e034"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:11:40.921576 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.921523 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-util" (OuterVolumeSpecName: "util") pod "fdceed12-3f3c-4c8b-ad29-03c60e25e034" (UID: "fdceed12-3f3c-4c8b-ad29-03c60e25e034"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:11:40.923698 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:40.923670 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdceed12-3f3c-4c8b-ad29-03c60e25e034-kube-api-access-78bnc" (OuterVolumeSpecName: "kube-api-access-78bnc") pod "fdceed12-3f3c-4c8b-ad29-03c60e25e034" (UID: "fdceed12-3f3c-4c8b-ad29-03c60e25e034"). InnerVolumeSpecName "kube-api-access-78bnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:11:41.014969 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.014943 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78bnc\" (UniqueName: \"kubernetes.io/projected/fdceed12-3f3c-4c8b-ad29-03c60e25e034-kube-api-access-78bnc\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:41.015084 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.014972 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:41.015084 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.014982 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdceed12-3f3c-4c8b-ad29-03c60e25e034-util\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:41.694191 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.694101 2553 generic.go:356] "Generic (PLEG): container finished" podID="2094f4b5-883b-4788-ab66-e971a541002a" containerID="daa8b454ed8c9ef23b5da358bcc395db553286f73a01fbe18ded5afb580c7e27" exitCode=0 Jul 22 20:11:41.694191 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.694150 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" event={"ID":"2094f4b5-883b-4788-ab66-e971a541002a","Type":"ContainerDied","Data":"daa8b454ed8c9ef23b5da358bcc395db553286f73a01fbe18ded5afb580c7e27"} Jul 22 20:11:41.696139 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.696121 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" Jul 22 20:11:41.696279 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.696155 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs" event={"ID":"fdceed12-3f3c-4c8b-ad29-03c60e25e034","Type":"ContainerDied","Data":"66abbad674bd908235431636ba02757d53b6115567610fa2ea0b20474154cb23"} Jul 22 20:11:41.696279 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.696204 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66abbad674bd908235431636ba02757d53b6115567610fa2ea0b20474154cb23" Jul 22 20:11:41.842829 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.842801 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:41.871619 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.871575 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:41.923516 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.923479 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-kube-api-access-j6c8b\") pod \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " Jul 22 20:11:41.923686 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.923542 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-bundle\") pod \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " Jul 22 20:11:41.923686 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.923576 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-util\") pod \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " Jul 22 20:11:41.923686 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.923603 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-util\") pod \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " Jul 22 20:11:41.923686 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.923636 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6j67\" (UniqueName: \"kubernetes.io/projected/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-kube-api-access-q6j67\") pod \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\" (UID: \"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500\") " Jul 22 20:11:41.923686 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.923661 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-bundle\") pod \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\" (UID: \"aa322f41-c382-4b42-9a9a-e2e9258aa5b0\") " Jul 22 20:11:41.924189 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.924156 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-bundle" (OuterVolumeSpecName: "bundle") pod "4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" (UID: "4f6d69b8-8d18-4ad1-9fa0-c82bf8604500"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:11:41.924487 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.924454 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-bundle" (OuterVolumeSpecName: "bundle") pod "aa322f41-c382-4b42-9a9a-e2e9258aa5b0" (UID: "aa322f41-c382-4b42-9a9a-e2e9258aa5b0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:11:41.926084 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.926053 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-kube-api-access-j6c8b" (OuterVolumeSpecName: "kube-api-access-j6c8b") pod "aa322f41-c382-4b42-9a9a-e2e9258aa5b0" (UID: "aa322f41-c382-4b42-9a9a-e2e9258aa5b0"). InnerVolumeSpecName "kube-api-access-j6c8b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:11:41.926084 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.926067 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-kube-api-access-q6j67" (OuterVolumeSpecName: "kube-api-access-q6j67") pod "4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" (UID: "4f6d69b8-8d18-4ad1-9fa0-c82bf8604500"). InnerVolumeSpecName "kube-api-access-q6j67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:11:41.929946 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.929907 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-util" (OuterVolumeSpecName: "util") pod "4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" (UID: "4f6d69b8-8d18-4ad1-9fa0-c82bf8604500"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:11:41.930643 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:41.930595 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-util" (OuterVolumeSpecName: "util") pod "aa322f41-c382-4b42-9a9a-e2e9258aa5b0" (UID: "aa322f41-c382-4b42-9a9a-e2e9258aa5b0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:11:42.024790 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.024752 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-kube-api-access-j6c8b\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:42.024790 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.024786 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:42.024790 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.024799 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-util\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:42.025017 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.024813 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-util\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:42.025017 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.024822 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6j67\" (UniqueName: \"kubernetes.io/projected/4f6d69b8-8d18-4ad1-9fa0-c82bf8604500-kube-api-access-q6j67\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:42.025017 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.024832 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa322f41-c382-4b42-9a9a-e2e9258aa5b0-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:42.703326 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.703244 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" Jul 22 20:11:42.703726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.703248 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz" event={"ID":"aa322f41-c382-4b42-9a9a-e2e9258aa5b0","Type":"ContainerDied","Data":"cc1b42effc63c5e01733f9dab512548fcdd02cf49e31e65721405333e424c2cc"} Jul 22 20:11:42.703726 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.703385 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc1b42effc63c5e01733f9dab512548fcdd02cf49e31e65721405333e424c2cc" Jul 22 20:11:42.705552 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.705530 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" Jul 22 20:11:42.705741 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.705574 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25" event={"ID":"4f6d69b8-8d18-4ad1-9fa0-c82bf8604500","Type":"ContainerDied","Data":"ea690fc91ec50986a6faeb8ee8ab7de994ef5ef5437028130492ffb0d0dd44e8"} Jul 22 20:11:42.705741 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.705614 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea690fc91ec50986a6faeb8ee8ab7de994ef5ef5437028130492ffb0d0dd44e8" Jul 22 20:11:42.858238 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.858208 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:42.933548 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.933503 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-util\") pod \"2094f4b5-883b-4788-ab66-e971a541002a\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " Jul 22 20:11:42.933763 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.933579 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-bundle\") pod \"2094f4b5-883b-4788-ab66-e971a541002a\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " Jul 22 20:11:42.933763 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.933631 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvlqn\" (UniqueName: \"kubernetes.io/projected/2094f4b5-883b-4788-ab66-e971a541002a-kube-api-access-tvlqn\") pod \"2094f4b5-883b-4788-ab66-e971a541002a\" (UID: \"2094f4b5-883b-4788-ab66-e971a541002a\") " Jul 22 20:11:42.934173 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.934146 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-bundle" (OuterVolumeSpecName: "bundle") pod "2094f4b5-883b-4788-ab66-e971a541002a" (UID: "2094f4b5-883b-4788-ab66-e971a541002a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:11:42.935829 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.935793 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2094f4b5-883b-4788-ab66-e971a541002a-kube-api-access-tvlqn" (OuterVolumeSpecName: "kube-api-access-tvlqn") pod "2094f4b5-883b-4788-ab66-e971a541002a" (UID: "2094f4b5-883b-4788-ab66-e971a541002a"). InnerVolumeSpecName "kube-api-access-tvlqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:11:42.939207 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:42.939176 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-util" (OuterVolumeSpecName: "util") pod "2094f4b5-883b-4788-ab66-e971a541002a" (UID: "2094f4b5-883b-4788-ab66-e971a541002a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:11:43.035008 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:43.034974 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvlqn\" (UniqueName: \"kubernetes.io/projected/2094f4b5-883b-4788-ab66-e971a541002a-kube-api-access-tvlqn\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:43.035008 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:43.035006 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-util\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:43.035008 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:43.035020 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2094f4b5-883b-4788-ab66-e971a541002a-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:11:43.713479 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:43.713440 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" event={"ID":"2094f4b5-883b-4788-ab66-e971a541002a","Type":"ContainerDied","Data":"14375f4bf6fcf6b3b614dd599926da32b02a508cd8e7b7baec96bf8e80663a6f"} Jul 22 20:11:43.713479 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:43.713469 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n" Jul 22 20:11:43.713912 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:43.713517 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14375f4bf6fcf6b3b614dd599926da32b02a508cd8e7b7baec96bf8e80663a6f" Jul 22 20:11:54.599656 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.599561 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bc6b4fb58-6czj2"] Jul 22 20:11:54.600027 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.599996 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerName="pull" Jul 22 20:11:54.600027 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600010 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerName="pull" Jul 22 20:11:54.600027 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600021 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2094f4b5-883b-4788-ab66-e971a541002a" containerName="pull" Jul 22 20:11:54.600027 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600028 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="2094f4b5-883b-4788-ab66-e971a541002a" containerName="pull" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600038 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2094f4b5-883b-4788-ab66-e971a541002a" containerName="extract" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600047 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="2094f4b5-883b-4788-ab66-e971a541002a" containerName="extract" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600056 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerName="util" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600061 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerName="util" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600068 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerName="util" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600074 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerName="util" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600083 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerName="extract" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600089 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerName="extract" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600095 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerName="pull" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600100 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerName="pull" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600105 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerName="extract" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600113 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerName="extract" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600125 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2094f4b5-883b-4788-ab66-e971a541002a" containerName="util" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600132 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="2094f4b5-883b-4788-ab66-e971a541002a" containerName="util" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600140 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerName="pull" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600145 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerName="pull" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600157 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerName="util" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600164 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerName="util" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600171 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerName="extract" Jul 22 20:11:54.600179 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600176 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerName="extract" Jul 22 20:11:54.601005 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600252 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f6d69b8-8d18-4ad1-9fa0-c82bf8604500" containerName="extract" Jul 22 20:11:54.601005 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600263 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa322f41-c382-4b42-9a9a-e2e9258aa5b0" containerName="extract" Jul 22 20:11:54.601005 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600273 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="2094f4b5-883b-4788-ab66-e971a541002a" containerName="extract" Jul 22 20:11:54.601005 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.600283 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdceed12-3f3c-4c8b-ad29-03c60e25e034" containerName="extract" Jul 22 20:11:54.603704 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.603660 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.625924 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.625887 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bc6b4fb58-6czj2"] Jul 22 20:11:54.745274 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.745228 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-console-config\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.745274 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.745275 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvcjm\" (UniqueName: \"kubernetes.io/projected/40665578-fee1-4407-bfae-3567534ba3c2-kube-api-access-qvcjm\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.745524 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.745296 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-service-ca\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.745524 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.745396 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40665578-fee1-4407-bfae-3567534ba3c2-console-serving-cert\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.745524 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.745471 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-trusted-ca-bundle\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.745524 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.745512 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40665578-fee1-4407-bfae-3567534ba3c2-console-oauth-config\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.745688 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.745531 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-oauth-serving-cert\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.846879 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.846832 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-console-config\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.846879 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.846880 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvcjm\" (UniqueName: \"kubernetes.io/projected/40665578-fee1-4407-bfae-3567534ba3c2-kube-api-access-qvcjm\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.847154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.846897 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-service-ca\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.847154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.846933 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40665578-fee1-4407-bfae-3567534ba3c2-console-serving-cert\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.847154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.846951 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-trusted-ca-bundle\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.847154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.846982 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40665578-fee1-4407-bfae-3567534ba3c2-console-oauth-config\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.847384 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.847147 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-oauth-serving-cert\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.847809 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.847783 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-console-config\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.847905 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.847833 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-oauth-serving-cert\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.847905 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.847834 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-service-ca\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.848019 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.847996 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40665578-fee1-4407-bfae-3567534ba3c2-trusted-ca-bundle\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.849589 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.849565 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/40665578-fee1-4407-bfae-3567534ba3c2-console-serving-cert\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.849716 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.849591 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/40665578-fee1-4407-bfae-3567534ba3c2-console-oauth-config\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.866102 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.866071 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvcjm\" (UniqueName: \"kubernetes.io/projected/40665578-fee1-4407-bfae-3567534ba3c2-kube-api-access-qvcjm\") pod \"console-5bc6b4fb58-6czj2\" (UID: \"40665578-fee1-4407-bfae-3567534ba3c2\") " pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.914743 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.914698 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:11:54.990307 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.990273 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg"] Jul 22 20:11:54.994072 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.994051 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" Jul 22 20:11:54.998775 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:54.998745 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-6ksf9\"" Jul 22 20:11:55.018756 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.018724 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg"] Jul 22 20:11:55.049906 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.049861 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wj64\" (UniqueName: \"kubernetes.io/projected/3348f631-5957-4bfe-ac51-428961c97785-kube-api-access-8wj64\") pod \"authorino-operator-75cc8d5cb7-2ftmg\" (UID: \"3348f631-5957-4bfe-ac51-428961c97785\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" Jul 22 20:11:55.102265 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.102233 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bc6b4fb58-6czj2"] Jul 22 20:11:55.104219 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:11:55.104179 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40665578_fee1_4407_bfae_3567534ba3c2.slice/crio-0ef453c18821f2c8e7cbb612f72fd592aa3f2c51400f0b18863d35a1373e9246 WatchSource:0}: Error finding container 0ef453c18821f2c8e7cbb612f72fd592aa3f2c51400f0b18863d35a1373e9246: Status 404 returned error can't find the container with id 0ef453c18821f2c8e7cbb612f72fd592aa3f2c51400f0b18863d35a1373e9246 Jul 22 20:11:55.151385 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.151345 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wj64\" (UniqueName: \"kubernetes.io/projected/3348f631-5957-4bfe-ac51-428961c97785-kube-api-access-8wj64\") pod \"authorino-operator-75cc8d5cb7-2ftmg\" (UID: \"3348f631-5957-4bfe-ac51-428961c97785\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" Jul 22 20:11:55.175098 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.175064 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wj64\" (UniqueName: \"kubernetes.io/projected/3348f631-5957-4bfe-ac51-428961c97785-kube-api-access-8wj64\") pod \"authorino-operator-75cc8d5cb7-2ftmg\" (UID: \"3348f631-5957-4bfe-ac51-428961c97785\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" Jul 22 20:11:55.317453 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.317387 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" Jul 22 20:11:55.476587 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.476559 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg"] Jul 22 20:11:55.478715 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:11:55.478684 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3348f631_5957_4bfe_ac51_428961c97785.slice/crio-5433514be0c7aff3552c010e76212af9bf352583f5a8aa8db26d8df8936cb3f9 WatchSource:0}: Error finding container 5433514be0c7aff3552c010e76212af9bf352583f5a8aa8db26d8df8936cb3f9: Status 404 returned error can't find the container with id 5433514be0c7aff3552c010e76212af9bf352583f5a8aa8db26d8df8936cb3f9 Jul 22 20:11:55.481030 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.481010 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 20:11:55.782187 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.782148 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" event={"ID":"3348f631-5957-4bfe-ac51-428961c97785","Type":"ContainerStarted","Data":"5433514be0c7aff3552c010e76212af9bf352583f5a8aa8db26d8df8936cb3f9"} Jul 22 20:11:55.783918 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.783893 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bc6b4fb58-6czj2" event={"ID":"40665578-fee1-4407-bfae-3567534ba3c2","Type":"ContainerStarted","Data":"712ea1e5284a89f3645d9adaed243e554ef74b418bfdb1d87f1672a9637d6046"} Jul 22 20:11:55.784033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.783936 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bc6b4fb58-6czj2" event={"ID":"40665578-fee1-4407-bfae-3567534ba3c2","Type":"ContainerStarted","Data":"0ef453c18821f2c8e7cbb612f72fd592aa3f2c51400f0b18863d35a1373e9246"} Jul 22 20:11:55.808618 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:55.808555 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bc6b4fb58-6czj2" podStartSLOduration=1.808536468 podStartE2EDuration="1.808536468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:11:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:11:55.807340934 +0000 UTC m=+625.364084628" watchObservedRunningTime="2026-07-22 20:11:55.808536468 +0000 UTC m=+625.365280163" Jul 22 20:11:58.175922 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.175871 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms"] Jul 22 20:11:58.179137 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.179104 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" Jul 22 20:11:58.181508 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.181468 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-59j8r\"" Jul 22 20:11:58.181701 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.181522 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 22 20:11:58.192102 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.192070 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms"] Jul 22 20:11:58.283742 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.283705 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kjsv\" (UniqueName: \"kubernetes.io/projected/ae794a3f-1e94-4d46-a267-20384eb124a5-kube-api-access-5kjsv\") pod \"dns-operator-controller-manager-864dd78c4c-bdqms\" (UID: \"ae794a3f-1e94-4d46-a267-20384eb124a5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" Jul 22 20:11:58.384940 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.384887 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kjsv\" (UniqueName: \"kubernetes.io/projected/ae794a3f-1e94-4d46-a267-20384eb124a5-kube-api-access-5kjsv\") pod \"dns-operator-controller-manager-864dd78c4c-bdqms\" (UID: \"ae794a3f-1e94-4d46-a267-20384eb124a5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" Jul 22 20:11:58.394529 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.394486 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kjsv\" (UniqueName: \"kubernetes.io/projected/ae794a3f-1e94-4d46-a267-20384eb124a5-kube-api-access-5kjsv\") pod \"dns-operator-controller-manager-864dd78c4c-bdqms\" (UID: \"ae794a3f-1e94-4d46-a267-20384eb124a5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" Jul 22 20:11:58.494643 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.494600 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" Jul 22 20:11:58.661796 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.661751 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms"] Jul 22 20:11:58.665059 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:11:58.664994 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae794a3f_1e94_4d46_a267_20384eb124a5.slice/crio-51320aa0287cacde8e0154147895202c0d93b5c565e39924291ba90c54217ed5 WatchSource:0}: Error finding container 51320aa0287cacde8e0154147895202c0d93b5c565e39924291ba90c54217ed5: Status 404 returned error can't find the container with id 51320aa0287cacde8e0154147895202c0d93b5c565e39924291ba90c54217ed5 Jul 22 20:11:58.810111 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.810003 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" event={"ID":"3348f631-5957-4bfe-ac51-428961c97785","Type":"ContainerStarted","Data":"c799d6e0c59059453a5088ebe7d4833257eb015d70b5aa2a8445ef623b885ff5"} Jul 22 20:11:58.810286 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.810178 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" Jul 22 20:11:58.811472 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.811438 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" event={"ID":"ae794a3f-1e94-4d46-a267-20384eb124a5","Type":"ContainerStarted","Data":"51320aa0287cacde8e0154147895202c0d93b5c565e39924291ba90c54217ed5"} Jul 22 20:11:58.848640 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:11:58.848581 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" podStartSLOduration=2.142045451 podStartE2EDuration="4.848558985s" totalImagesPullingTime="2.706513534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:11:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:11:58.846548663 +0000 UTC m=+628.403292380" watchObservedRunningTime="2026-07-22 20:11:58.848558985 +0000 UTC m=+628.405302680" Jul 22 20:12:00.831054 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:00.831000 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" event={"ID":"ae794a3f-1e94-4d46-a267-20384eb124a5","Type":"ContainerStarted","Data":"455ec3edc72913487d54952fa00154a20972a6f44df30361efcff6767fd68da2"} Jul 22 20:12:00.831559 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:00.831074 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" Jul 22 20:12:00.852848 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:00.852776 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" podStartSLOduration=0.904669763 podStartE2EDuration="2.852757442s" totalImagesPullingTime="1.948087679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:12:00.849694461 +0000 UTC m=+630.406438154" watchObservedRunningTime="2026-07-22 20:12:00.852757442 +0000 UTC m=+630.409501215" Jul 22 20:12:01.627154 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:01.627114 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n"] Jul 22 20:12:01.631474 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:01.631445 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:01.634123 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:01.634090 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-275qv\"" Jul 22 20:12:01.645373 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:01.645329 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n"] Jul 22 20:12:01.718568 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:01.718517 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdwsq\" (UniqueName: \"kubernetes.io/projected/9a2f54fd-7754-4d42-a760-e3c44d1761e4-kube-api-access-gdwsq\") pod \"limitador-operator-controller-manager-859d7b78bd-hjb4n\" (UID: \"9a2f54fd-7754-4d42-a760-e3c44d1761e4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:01.819124 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:01.819073 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdwsq\" (UniqueName: \"kubernetes.io/projected/9a2f54fd-7754-4d42-a760-e3c44d1761e4-kube-api-access-gdwsq\") pod \"limitador-operator-controller-manager-859d7b78bd-hjb4n\" (UID: \"9a2f54fd-7754-4d42-a760-e3c44d1761e4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:01.844055 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:01.844015 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdwsq\" (UniqueName: \"kubernetes.io/projected/9a2f54fd-7754-4d42-a760-e3c44d1761e4-kube-api-access-gdwsq\") pod \"limitador-operator-controller-manager-859d7b78bd-hjb4n\" (UID: \"9a2f54fd-7754-4d42-a760-e3c44d1761e4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:01.944547 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:01.944406 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:02.102893 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:02.102856 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n"] Jul 22 20:12:02.104618 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:12:02.104586 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a2f54fd_7754_4d42_a760_e3c44d1761e4.slice/crio-f43b923082e53b66062a057e2bbc508157cabb63a3ea367a3f978701d47907ec WatchSource:0}: Error finding container f43b923082e53b66062a057e2bbc508157cabb63a3ea367a3f978701d47907ec: Status 404 returned error can't find the container with id f43b923082e53b66062a057e2bbc508157cabb63a3ea367a3f978701d47907ec Jul 22 20:12:02.858468 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:02.858392 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" event={"ID":"9a2f54fd-7754-4d42-a760-e3c44d1761e4","Type":"ContainerStarted","Data":"f43b923082e53b66062a057e2bbc508157cabb63a3ea367a3f978701d47907ec"} Jul 22 20:12:04.881004 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:04.880953 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" event={"ID":"9a2f54fd-7754-4d42-a760-e3c44d1761e4","Type":"ContainerStarted","Data":"8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9"} Jul 22 20:12:04.881455 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:04.881161 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:04.901210 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:04.901149 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" podStartSLOduration=1.876083628 podStartE2EDuration="3.901130551s" totalImagesPullingTime="2.025046923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:12:04.899872169 +0000 UTC m=+634.456615864" watchObservedRunningTime="2026-07-22 20:12:04.901130551 +0000 UTC m=+634.457874245" Jul 22 20:12:04.915374 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:04.915315 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:12:04.915545 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:04.915404 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:12:04.920825 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:04.920796 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:12:05.891450 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:05.891395 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bc6b4fb58-6czj2" Jul 22 20:12:05.958043 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:05.958003 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-596db6f779-8mhwf"] Jul 22 20:12:06.538572 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.538530 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p"] Jul 22 20:12:06.542545 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.542521 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:06.546354 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.546322 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rjs97\"" Jul 22 20:12:06.555140 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.555099 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p"] Jul 22 20:12:06.668143 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.668097 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf5b3879-2d82-4b05-aaf3-6250285595c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" (UID: \"bf5b3879-2d82-4b05-aaf3-6250285595c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:06.668315 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.668179 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frdqw\" (UniqueName: \"kubernetes.io/projected/bf5b3879-2d82-4b05-aaf3-6250285595c9-kube-api-access-frdqw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" (UID: \"bf5b3879-2d82-4b05-aaf3-6250285595c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:06.769116 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.769068 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf5b3879-2d82-4b05-aaf3-6250285595c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" (UID: \"bf5b3879-2d82-4b05-aaf3-6250285595c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:06.769313 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.769142 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frdqw\" (UniqueName: \"kubernetes.io/projected/bf5b3879-2d82-4b05-aaf3-6250285595c9-kube-api-access-frdqw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" (UID: \"bf5b3879-2d82-4b05-aaf3-6250285595c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:06.769700 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.769669 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf5b3879-2d82-4b05-aaf3-6250285595c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" (UID: \"bf5b3879-2d82-4b05-aaf3-6250285595c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:06.780494 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.780461 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frdqw\" (UniqueName: \"kubernetes.io/projected/bf5b3879-2d82-4b05-aaf3-6250285595c9-kube-api-access-frdqw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" (UID: \"bf5b3879-2d82-4b05-aaf3-6250285595c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:06.856873 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:06.856771 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:07.018749 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:07.018714 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p"] Jul 22 20:12:07.021134 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:12:07.021093 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf5b3879_2d82_4b05_aaf3_6250285595c9.slice/crio-ce695142b7038b8e7948553bf8114fc0b2fe2f59a7086a72bca51fcffaacaa14 WatchSource:0}: Error finding container ce695142b7038b8e7948553bf8114fc0b2fe2f59a7086a72bca51fcffaacaa14: Status 404 returned error can't find the container with id ce695142b7038b8e7948553bf8114fc0b2fe2f59a7086a72bca51fcffaacaa14 Jul 22 20:12:07.900362 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:07.900324 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" event={"ID":"bf5b3879-2d82-4b05-aaf3-6250285595c9","Type":"ContainerStarted","Data":"ce695142b7038b8e7948553bf8114fc0b2fe2f59a7086a72bca51fcffaacaa14"} Jul 22 20:12:09.826258 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:09.826224 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2ftmg" Jul 22 20:12:11.840828 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:11.840790 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bdqms" Jul 22 20:12:14.949773 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:14.949719 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" event={"ID":"bf5b3879-2d82-4b05-aaf3-6250285595c9","Type":"ContainerStarted","Data":"311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad"} Jul 22 20:12:14.950194 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:14.949824 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:14.973863 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:14.973786 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" podStartSLOduration=1.583626768 podStartE2EDuration="8.973758265s" totalImagesPullingTime="7.390131497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:12:14.972497699 +0000 UTC m=+644.529241397" watchObservedRunningTime="2026-07-22 20:12:14.973758265 +0000 UTC m=+644.530501964" Jul 22 20:12:15.889953 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:15.889915 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:17.573237 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.573190 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd"] Jul 22 20:12:17.577450 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.577401 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:17.579902 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.579866 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x4sxh\"" Jul 22 20:12:17.580098 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.579863 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 22 20:12:17.580098 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.579873 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 22 20:12:17.587065 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.587031 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd"] Jul 22 20:12:17.669363 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.669313 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e14b33a-43c9-4a0d-8487-661c149da3c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:17.669603 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.669453 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx82g\" (UniqueName: \"kubernetes.io/projected/2e14b33a-43c9-4a0d-8487-661c149da3c1-kube-api-access-kx82g\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:17.669603 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.669516 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e14b33a-43c9-4a0d-8487-661c149da3c1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:17.770689 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.770640 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e14b33a-43c9-4a0d-8487-661c149da3c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:17.770901 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.770725 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx82g\" (UniqueName: \"kubernetes.io/projected/2e14b33a-43c9-4a0d-8487-661c149da3c1-kube-api-access-kx82g\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:17.770901 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.770771 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e14b33a-43c9-4a0d-8487-661c149da3c1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:17.770901 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:17.770798 2553 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 22 20:12:17.770901 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:17.770875 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e14b33a-43c9-4a0d-8487-661c149da3c1-plugin-serving-cert podName:2e14b33a-43c9-4a0d-8487-661c149da3c1 nodeName:}" failed. No retries permitted until 2026-07-22 20:12:18.270851913 +0000 UTC m=+647.827595586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/2e14b33a-43c9-4a0d-8487-661c149da3c1-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-4ltjd" (UID: "2e14b33a-43c9-4a0d-8487-661c149da3c1") : secret "plugin-serving-cert" not found Jul 22 20:12:17.771477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.771458 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e14b33a-43c9-4a0d-8487-661c149da3c1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:17.784424 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:17.784381 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx82g\" (UniqueName: \"kubernetes.io/projected/2e14b33a-43c9-4a0d-8487-661c149da3c1-kube-api-access-kx82g\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:18.274670 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:18.274585 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e14b33a-43c9-4a0d-8487-661c149da3c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:18.278980 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:18.278911 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e14b33a-43c9-4a0d-8487-661c149da3c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4ltjd\" (UID: \"2e14b33a-43c9-4a0d-8487-661c149da3c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:18.490239 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:18.490194 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" Jul 22 20:12:18.646891 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:18.646862 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd"] Jul 22 20:12:18.649048 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:12:18.649013 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e14b33a_43c9_4a0d_8487_661c149da3c1.slice/crio-76b34b92dc40c9406584a13db3578027f6eb88a9ad22b52b100ca1b4802cd780 WatchSource:0}: Error finding container 76b34b92dc40c9406584a13db3578027f6eb88a9ad22b52b100ca1b4802cd780: Status 404 returned error can't find the container with id 76b34b92dc40c9406584a13db3578027f6eb88a9ad22b52b100ca1b4802cd780 Jul 22 20:12:18.979610 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:18.979568 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" event={"ID":"2e14b33a-43c9-4a0d-8487-661c149da3c1","Type":"ContainerStarted","Data":"76b34b92dc40c9406584a13db3578027f6eb88a9ad22b52b100ca1b4802cd780"} Jul 22 20:12:25.961124 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:25.961088 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:27.598895 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.598848 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z"] Jul 22 20:12:27.605878 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.605842 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:27.617457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.617384 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z"] Jul 22 20:12:27.671889 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.671834 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/819deea3-aaec-475a-8c0b-e46a881aff37-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" (UID: \"819deea3-aaec-475a-8c0b-e46a881aff37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:27.672106 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.671981 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pp9x\" (UniqueName: \"kubernetes.io/projected/819deea3-aaec-475a-8c0b-e46a881aff37-kube-api-access-6pp9x\") pod \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" (UID: \"819deea3-aaec-475a-8c0b-e46a881aff37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:27.773254 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.773202 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/819deea3-aaec-475a-8c0b-e46a881aff37-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" (UID: \"819deea3-aaec-475a-8c0b-e46a881aff37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:27.773489 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.773316 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pp9x\" (UniqueName: \"kubernetes.io/projected/819deea3-aaec-475a-8c0b-e46a881aff37-kube-api-access-6pp9x\") pod \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" (UID: \"819deea3-aaec-475a-8c0b-e46a881aff37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:27.773724 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.773687 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/819deea3-aaec-475a-8c0b-e46a881aff37-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" (UID: \"819deea3-aaec-475a-8c0b-e46a881aff37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:27.786652 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.786616 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pp9x\" (UniqueName: \"kubernetes.io/projected/819deea3-aaec-475a-8c0b-e46a881aff37-kube-api-access-6pp9x\") pod \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" (UID: \"819deea3-aaec-475a-8c0b-e46a881aff37\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:27.810894 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.810850 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z"] Jul 22 20:12:27.811257 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.811235 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:27.829330 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.829287 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z"] Jul 22 20:12:27.846947 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.846911 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z"] Jul 22 20:12:27.862466 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.862164 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p"] Jul 22 20:12:27.862837 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.862799 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" containerName="manager" containerID="cri-o://311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad" gracePeriod=2 Jul 22 20:12:27.866209 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:27.866169 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:27.866594 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.866565 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p"] Jul 22 20:12:27.885331 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.885296 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n"] Jul 22 20:12:27.885621 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.885569 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" containerName="manager" containerID="cri-o://8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9" gracePeriod=2 Jul 22 20:12:27.897205 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.897150 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9"] Jul 22 20:12:27.898127 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.897975 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" containerName="manager" Jul 22 20:12:27.898127 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.898008 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" containerName="manager" Jul 22 20:12:27.898127 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.898134 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" containerName="manager" Jul 22 20:12:27.902189 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.902157 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:27.904464 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.904431 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n"] Jul 22 20:12:27.919745 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:27.919642 2553 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9a2f54fd-7754-4d42-a760-e3c44d1761e4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-hjb4n\": pods \"limitador-operator-controller-manager-859d7b78bd-hjb4n\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:27.922488 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:27.922404 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:27.923598 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.923521 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9"] Jul 22 20:12:27.924962 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:27.924630 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:27.930125 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.930090 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc"] Jul 22 20:12:27.930799 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.930775 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" containerName="manager" Jul 22 20:12:27.930799 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.930800 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" containerName="manager" Jul 22 20:12:27.931017 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.930931 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" containerName="manager" Jul 22 20:12:27.934884 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.934862 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" Jul 22 20:12:27.955753 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.955705 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc"] Jul 22 20:12:27.958555 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:27.958511 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hjb4n\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:27.961840 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:27.961799 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hjb4n\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:27.975739 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.975703 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k526h\" (UniqueName: \"kubernetes.io/projected/6fb5da73-00d8-4d50-a0b2-e0c6806102e9-kube-api-access-k526h\") pod \"limitador-operator-controller-manager-859d7b78bd-nlwsc\" (UID: \"6fb5da73-00d8-4d50-a0b2-e0c6806102e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" Jul 22 20:12:27.975898 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.975756 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89db54bc-657c-4d1c-8f60-d8edef3dde39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prrv9\" (UID: \"89db54bc-657c-4d1c-8f60-d8edef3dde39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:27.975973 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:27.975901 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bjqr\" (UniqueName: \"kubernetes.io/projected/89db54bc-657c-4d1c-8f60-d8edef3dde39-kube-api-access-9bjqr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prrv9\" (UID: \"89db54bc-657c-4d1c-8f60-d8edef3dde39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:27.983698 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:27.983652 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:28.076982 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:28.076930 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bjqr\" (UniqueName: \"kubernetes.io/projected/89db54bc-657c-4d1c-8f60-d8edef3dde39-kube-api-access-9bjqr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prrv9\" (UID: \"89db54bc-657c-4d1c-8f60-d8edef3dde39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:28.077200 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:28.076996 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k526h\" (UniqueName: \"kubernetes.io/projected/6fb5da73-00d8-4d50-a0b2-e0c6806102e9-kube-api-access-k526h\") pod \"limitador-operator-controller-manager-859d7b78bd-nlwsc\" (UID: \"6fb5da73-00d8-4d50-a0b2-e0c6806102e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" Jul 22 20:12:28.077200 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:28.077030 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89db54bc-657c-4d1c-8f60-d8edef3dde39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prrv9\" (UID: \"89db54bc-657c-4d1c-8f60-d8edef3dde39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:28.077627 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:28.077601 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89db54bc-657c-4d1c-8f60-d8edef3dde39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prrv9\" (UID: \"89db54bc-657c-4d1c-8f60-d8edef3dde39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:28.089917 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:28.089847 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bjqr\" (UniqueName: \"kubernetes.io/projected/89db54bc-657c-4d1c-8f60-d8edef3dde39-kube-api-access-9bjqr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prrv9\" (UID: \"89db54bc-657c-4d1c-8f60-d8edef3dde39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:28.089917 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:28.089912 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k526h\" (UniqueName: \"kubernetes.io/projected/6fb5da73-00d8-4d50-a0b2-e0c6806102e9-kube-api-access-k526h\") pod \"limitador-operator-controller-manager-859d7b78bd-nlwsc\" (UID: \"6fb5da73-00d8-4d50-a0b2-e0c6806102e9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" Jul 22 20:12:28.288942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:28.288611 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:28.296764 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:28.296728 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" Jul 22 20:12:30.981826 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:30.981775 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-596db6f779-8mhwf" podUID="4a897897-2dbe-42d7-bd18-0c85f73ee0fc" containerName="console" containerID="cri-o://8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c" gracePeriod=15 Jul 22 20:12:39.462396 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:39.462348 2553 patch_prober.go:28] interesting pod/console-596db6f779-8mhwf container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused" start-of-body= Jul 22 20:12:39.462964 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:39.462457 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-596db6f779-8mhwf" podUID="4a897897-2dbe-42d7-bd18-0c85f73ee0fc" containerName="console" probeResult="failure" output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused" Jul 22 20:12:42.608347 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.608309 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:12:42.621604 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.621572 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc"] Jul 22 20:12:42.625767 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:12:42.625614 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb5da73_00d8_4d50_a0b2_e0c6806102e9.slice/crio-1cd4b6156450526c01e7853121ae496417615425d5224bdc61d9880ffd15c09f WatchSource:0}: Error finding container 1cd4b6156450526c01e7853121ae496417615425d5224bdc61d9880ffd15c09f: Status 404 returned error can't find the container with id 1cd4b6156450526c01e7853121ae496417615425d5224bdc61d9880ffd15c09f Jul 22 20:12:42.627277 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.627228 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:42.631443 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:42.631386 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hjb4n\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:42.636477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.636446 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:42.638292 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:42.638242 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:42.639938 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:42.639902 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hjb4n\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:42.655986 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.655961 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9"] Jul 22 20:12:42.660642 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:12:42.660615 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89db54bc_657c_4d1c_8f60_d8edef3dde39.slice/crio-3cf39c76ab3f21dfd02ba2de43b75b88c040f376c1f018c9bd41eb2d32e8a069 WatchSource:0}: Error finding container 3cf39c76ab3f21dfd02ba2de43b75b88c040f376c1f018c9bd41eb2d32e8a069: Status 404 returned error can't find the container with id 3cf39c76ab3f21dfd02ba2de43b75b88c040f376c1f018c9bd41eb2d32e8a069 Jul 22 20:12:42.741568 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741525 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-config\") pod \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " Jul 22 20:12:42.741761 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741585 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-service-ca\") pod \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " Jul 22 20:12:42.741761 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741632 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf5b3879-2d82-4b05-aaf3-6250285595c9-extensions-socket-volume\") pod \"bf5b3879-2d82-4b05-aaf3-6250285595c9\" (UID: \"bf5b3879-2d82-4b05-aaf3-6250285595c9\") " Jul 22 20:12:42.741761 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741655 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frdqw\" (UniqueName: \"kubernetes.io/projected/bf5b3879-2d82-4b05-aaf3-6250285595c9-kube-api-access-frdqw\") pod \"bf5b3879-2d82-4b05-aaf3-6250285595c9\" (UID: \"bf5b3879-2d82-4b05-aaf3-6250285595c9\") " Jul 22 20:12:42.741761 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741694 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-oauth-serving-cert\") pod \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " Jul 22 20:12:42.741761 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741733 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-oauth-config\") pod \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " Jul 22 20:12:42.741761 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741759 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnbs7\" (UniqueName: \"kubernetes.io/projected/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-kube-api-access-vnbs7\") pod \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " Jul 22 20:12:42.742075 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741787 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-serving-cert\") pod \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " Jul 22 20:12:42.742075 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741817 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-trusted-ca-bundle\") pod \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\" (UID: \"4a897897-2dbe-42d7-bd18-0c85f73ee0fc\") " Jul 22 20:12:42.742075 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.741869 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdwsq\" (UniqueName: \"kubernetes.io/projected/9a2f54fd-7754-4d42-a760-e3c44d1761e4-kube-api-access-gdwsq\") pod \"9a2f54fd-7754-4d42-a760-e3c44d1761e4\" (UID: \"9a2f54fd-7754-4d42-a760-e3c44d1761e4\") " Jul 22 20:12:42.742075 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.742010 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-config" (OuterVolumeSpecName: "console-config") pod "4a897897-2dbe-42d7-bd18-0c85f73ee0fc" (UID: "4a897897-2dbe-42d7-bd18-0c85f73ee0fc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:12:42.742075 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.742056 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a897897-2dbe-42d7-bd18-0c85f73ee0fc" (UID: "4a897897-2dbe-42d7-bd18-0c85f73ee0fc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:12:42.742321 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.742263 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf5b3879-2d82-4b05-aaf3-6250285595c9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bf5b3879-2d82-4b05-aaf3-6250285595c9" (UID: "bf5b3879-2d82-4b05-aaf3-6250285595c9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:12:42.742321 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.742276 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-service-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:42.742321 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.742296 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-config\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:42.742519 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.742289 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a897897-2dbe-42d7-bd18-0c85f73ee0fc" (UID: "4a897897-2dbe-42d7-bd18-0c85f73ee0fc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:12:42.742578 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.742536 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4a897897-2dbe-42d7-bd18-0c85f73ee0fc" (UID: "4a897897-2dbe-42d7-bd18-0c85f73ee0fc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:12:42.744096 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.744065 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf5b3879-2d82-4b05-aaf3-6250285595c9-kube-api-access-frdqw" (OuterVolumeSpecName: "kube-api-access-frdqw") pod "bf5b3879-2d82-4b05-aaf3-6250285595c9" (UID: "bf5b3879-2d82-4b05-aaf3-6250285595c9"). InnerVolumeSpecName "kube-api-access-frdqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:12:42.744275 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.744243 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a897897-2dbe-42d7-bd18-0c85f73ee0fc" (UID: "4a897897-2dbe-42d7-bd18-0c85f73ee0fc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:12:42.744333 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.744299 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-kube-api-access-vnbs7" (OuterVolumeSpecName: "kube-api-access-vnbs7") pod "4a897897-2dbe-42d7-bd18-0c85f73ee0fc" (UID: "4a897897-2dbe-42d7-bd18-0c85f73ee0fc"). InnerVolumeSpecName "kube-api-access-vnbs7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:12:42.744612 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.744581 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a897897-2dbe-42d7-bd18-0c85f73ee0fc" (UID: "4a897897-2dbe-42d7-bd18-0c85f73ee0fc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:12:42.744612 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.744598 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a2f54fd-7754-4d42-a760-e3c44d1761e4-kube-api-access-gdwsq" (OuterVolumeSpecName: "kube-api-access-gdwsq") pod "9a2f54fd-7754-4d42-a760-e3c44d1761e4" (UID: "9a2f54fd-7754-4d42-a760-e3c44d1761e4"). InnerVolumeSpecName "kube-api-access-gdwsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:12:42.842746 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.842705 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-oauth-serving-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:42.842746 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.842738 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-oauth-config\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:42.842746 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.842747 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnbs7\" (UniqueName: \"kubernetes.io/projected/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-kube-api-access-vnbs7\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:42.842746 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.842756 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-console-serving-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:42.843021 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.842765 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a897897-2dbe-42d7-bd18-0c85f73ee0fc-trusted-ca-bundle\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:42.843021 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.842773 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdwsq\" (UniqueName: \"kubernetes.io/projected/9a2f54fd-7754-4d42-a760-e3c44d1761e4-kube-api-access-gdwsq\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:42.843021 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.842781 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf5b3879-2d82-4b05-aaf3-6250285595c9-extensions-socket-volume\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:42.843021 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:42.842789 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frdqw\" (UniqueName: \"kubernetes.io/projected/bf5b3879-2d82-4b05-aaf3-6250285595c9-kube-api-access-frdqw\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:43.007245 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.007209 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" path="/var/lib/kubelet/pods/9a2f54fd-7754-4d42-a760-e3c44d1761e4/volumes" Jul 22 20:12:43.007604 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.007582 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" path="/var/lib/kubelet/pods/bf5b3879-2d82-4b05-aaf3-6250285595c9/volumes" Jul 22 20:12:43.152498 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.152454 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" event={"ID":"89db54bc-657c-4d1c-8f60-d8edef3dde39","Type":"ContainerStarted","Data":"79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596"} Jul 22 20:12:43.152705 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.152552 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" event={"ID":"89db54bc-657c-4d1c-8f60-d8edef3dde39","Type":"ContainerStarted","Data":"3cf39c76ab3f21dfd02ba2de43b75b88c040f376c1f018c9bd41eb2d32e8a069"} Jul 22 20:12:43.152705 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.152582 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:43.154327 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.154294 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" event={"ID":"6fb5da73-00d8-4d50-a0b2-e0c6806102e9","Type":"ContainerStarted","Data":"6d41d0eabd9e381bb9ffb1e738dc835a831b96c239dbd25c406bc9a9e9688c94"} Jul 22 20:12:43.154476 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.154350 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" event={"ID":"6fb5da73-00d8-4d50-a0b2-e0c6806102e9","Type":"ContainerStarted","Data":"1cd4b6156450526c01e7853121ae496417615425d5224bdc61d9880ffd15c09f"} Jul 22 20:12:43.154476 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.154400 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" Jul 22 20:12:43.155806 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.155780 2553 generic.go:356] "Generic (PLEG): container finished" podID="4a897897-2dbe-42d7-bd18-0c85f73ee0fc" containerID="8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c" exitCode=0 Jul 22 20:12:43.155932 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.155827 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-596db6f779-8mhwf" Jul 22 20:12:43.155932 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.155857 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-596db6f779-8mhwf" event={"ID":"4a897897-2dbe-42d7-bd18-0c85f73ee0fc","Type":"ContainerDied","Data":"8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c"} Jul 22 20:12:43.155932 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.155894 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-596db6f779-8mhwf" event={"ID":"4a897897-2dbe-42d7-bd18-0c85f73ee0fc","Type":"ContainerDied","Data":"d67653e11d4d6509c2dfa7987b2b749dc671913f473466cb85b81876477d0269"} Jul 22 20:12:43.155932 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.155923 2553 scope.go:117] "RemoveContainer" containerID="8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c" Jul 22 20:12:43.157367 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.157343 2553 generic.go:356] "Generic (PLEG): container finished" podID="bf5b3879-2d82-4b05-aaf3-6250285595c9" containerID="311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad" exitCode=0 Jul 22 20:12:43.157495 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.157424 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:43.158966 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.158939 2553 generic.go:356] "Generic (PLEG): container finished" podID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" containerID="8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9" exitCode=0 Jul 22 20:12:43.159089 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.158984 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:43.161435 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.161393 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" event={"ID":"2e14b33a-43c9-4a0d-8487-661c149da3c1","Type":"ContainerStarted","Data":"9401d39ceaf4f48601e2ca977b0b8bb5cccf4b4e816ab8f97fcbcce40e2a1f80"} Jul 22 20:12:43.168053 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.168031 2553 scope.go:117] "RemoveContainer" containerID="8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c" Jul 22 20:12:43.168352 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:43.168332 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c\": container with ID starting with 8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c not found: ID does not exist" containerID="8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c" Jul 22 20:12:43.168404 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.168362 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c"} err="failed to get container status \"8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c\": rpc error: code = NotFound desc = could not find container \"8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c\": container with ID starting with 8c7a9c0f76cc27ce7a8e3489b76bb65d8693b09396acb513400020cd674a849c not found: ID does not exist" Jul 22 20:12:43.168404 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.168379 2553 scope.go:117] "RemoveContainer" containerID="311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad" Jul 22 20:12:43.176391 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.176345 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" podStartSLOduration=16.176330647 podStartE2EDuration="16.176330647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:12:43.172691426 +0000 UTC m=+672.729435122" watchObservedRunningTime="2026-07-22 20:12:43.176330647 +0000 UTC m=+672.733074342" Jul 22 20:12:43.178147 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.178132 2553 scope.go:117] "RemoveContainer" containerID="311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad" Jul 22 20:12:43.178465 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:43.178442 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad\": container with ID starting with 311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad not found: ID does not exist" containerID="311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad" Jul 22 20:12:43.178515 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.178474 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad"} err="failed to get container status \"311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad\": rpc error: code = NotFound desc = could not find container \"311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad\": container with ID starting with 311119cdf938b3bbd0b1488d089fce493c7d6652cc90bb83ee8a8e9b828462ad not found: ID does not exist" Jul 22 20:12:43.178515 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.178492 2553 scope.go:117] "RemoveContainer" containerID="8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9" Jul 22 20:12:43.187250 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.187227 2553 scope.go:117] "RemoveContainer" containerID="8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9" Jul 22 20:12:43.187572 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:43.187549 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9\": container with ID starting with 8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9 not found: ID does not exist" containerID="8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9" Jul 22 20:12:43.187633 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.187585 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9"} err="failed to get container status \"8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9\": rpc error: code = NotFound desc = could not find container \"8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9\": container with ID starting with 8c5bff1ed4dfa5a1f0f8147eb8ba03ab79da8da6e0d2b36591ae5cdda8cd2bc9 not found: ID does not exist" Jul 22 20:12:43.191022 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.190978 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" podStartSLOduration=16.19096281 podStartE2EDuration="16.19096281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:12:43.189205099 +0000 UTC m=+672.745948818" watchObservedRunningTime="2026-07-22 20:12:43.19096281 +0000 UTC m=+672.747706562" Jul 22 20:12:43.204345 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.204311 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-596db6f779-8mhwf"] Jul 22 20:12:43.211980 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.211952 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-596db6f779-8mhwf"] Jul 22 20:12:43.213472 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:43.213446 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-h9w2p\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="bf5b3879-2d82-4b05-aaf3-6250285595c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h9w2p" Jul 22 20:12:43.215185 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:43.215157 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hjb4n\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="9a2f54fd-7754-4d42-a760-e3c44d1761e4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hjb4n" Jul 22 20:12:43.236696 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:43.236646 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4ltjd" podStartSLOduration=2.379861537 podStartE2EDuration="26.236630694s" totalImagesPullingTime="23.856769157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:12:43.234880366 +0000 UTC m=+672.791624077" watchObservedRunningTime="2026-07-22 20:12:43.236630694 +0000 UTC m=+672.793374389" Jul 22 20:12:44.954330 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:44.954290 2553 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-nzm7z_kuadrant-system_819deea3-aaec-475a-8c0b-e46a881aff37_0(42c37a71c02d03b87e69f2f9c392293c20a65ca1f52c96af14fe203e1f0a67ca): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-nzm7z to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 22 20:12:44.954624 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:44.954355 2553 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-nzm7z_kuadrant-system_819deea3-aaec-475a-8c0b-e46a881aff37_0(42c37a71c02d03b87e69f2f9c392293c20a65ca1f52c96af14fe203e1f0a67ca): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-nzm7z to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:45.013338 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.013300 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a897897-2dbe-42d7-bd18-0c85f73ee0fc" path="/var/lib/kubelet/pods/4a897897-2dbe-42d7-bd18-0c85f73ee0fc/volumes" Jul 22 20:12:45.176463 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.176428 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:45.178399 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:45.178368 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="819deea3-aaec-475a-8c0b-e46a881aff37" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:45.186325 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.186299 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:45.188249 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:45.188198 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="819deea3-aaec-475a-8c0b-e46a881aff37" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:45.301342 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.301307 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9"] Jul 22 20:12:45.301555 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.301513 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" podUID="89db54bc-657c-4d1c-8f60-d8edef3dde39" containerName="manager" containerID="cri-o://79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596" gracePeriod=10 Jul 22 20:12:45.303549 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:45.303521 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="819deea3-aaec-475a-8c0b-e46a881aff37" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:45.369637 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.369600 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/819deea3-aaec-475a-8c0b-e46a881aff37-extensions-socket-volume\") pod \"819deea3-aaec-475a-8c0b-e46a881aff37\" (UID: \"819deea3-aaec-475a-8c0b-e46a881aff37\") " Jul 22 20:12:45.369831 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.369703 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pp9x\" (UniqueName: \"kubernetes.io/projected/819deea3-aaec-475a-8c0b-e46a881aff37-kube-api-access-6pp9x\") pod \"819deea3-aaec-475a-8c0b-e46a881aff37\" (UID: \"819deea3-aaec-475a-8c0b-e46a881aff37\") " Jul 22 20:12:45.369911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.369870 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819deea3-aaec-475a-8c0b-e46a881aff37-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "819deea3-aaec-475a-8c0b-e46a881aff37" (UID: "819deea3-aaec-475a-8c0b-e46a881aff37"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:12:45.370019 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.370003 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/819deea3-aaec-475a-8c0b-e46a881aff37-extensions-socket-volume\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:45.371807 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.371780 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/819deea3-aaec-475a-8c0b-e46a881aff37-kube-api-access-6pp9x" (OuterVolumeSpecName: "kube-api-access-6pp9x") pod "819deea3-aaec-475a-8c0b-e46a881aff37" (UID: "819deea3-aaec-475a-8c0b-e46a881aff37"). InnerVolumeSpecName "kube-api-access-6pp9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:12:45.471134 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.471099 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pp9x\" (UniqueName: \"kubernetes.io/projected/819deea3-aaec-475a-8c0b-e46a881aff37-kube-api-access-6pp9x\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:45.570251 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.570212 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:45.599618 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:45.599576 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="819deea3-aaec-475a-8c0b-e46a881aff37" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:45.672876 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.672844 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89db54bc-657c-4d1c-8f60-d8edef3dde39-extensions-socket-volume\") pod \"89db54bc-657c-4d1c-8f60-d8edef3dde39\" (UID: \"89db54bc-657c-4d1c-8f60-d8edef3dde39\") " Jul 22 20:12:45.673067 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.672951 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bjqr\" (UniqueName: \"kubernetes.io/projected/89db54bc-657c-4d1c-8f60-d8edef3dde39-kube-api-access-9bjqr\") pod \"89db54bc-657c-4d1c-8f60-d8edef3dde39\" (UID: \"89db54bc-657c-4d1c-8f60-d8edef3dde39\") " Jul 22 20:12:45.673242 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.673212 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89db54bc-657c-4d1c-8f60-d8edef3dde39-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "89db54bc-657c-4d1c-8f60-d8edef3dde39" (UID: "89db54bc-657c-4d1c-8f60-d8edef3dde39"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 20:12:45.674972 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.674945 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89db54bc-657c-4d1c-8f60-d8edef3dde39-kube-api-access-9bjqr" (OuterVolumeSpecName: "kube-api-access-9bjqr") pod "89db54bc-657c-4d1c-8f60-d8edef3dde39" (UID: "89db54bc-657c-4d1c-8f60-d8edef3dde39"). InnerVolumeSpecName "kube-api-access-9bjqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:12:45.774651 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.774559 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89db54bc-657c-4d1c-8f60-d8edef3dde39-extensions-socket-volume\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:45.774651 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:45.774597 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bjqr\" (UniqueName: \"kubernetes.io/projected/89db54bc-657c-4d1c-8f60-d8edef3dde39-kube-api-access-9bjqr\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:12:46.183603 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.183516 2553 generic.go:356] "Generic (PLEG): container finished" podID="89db54bc-657c-4d1c-8f60-d8edef3dde39" containerID="79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596" exitCode=0 Jul 22 20:12:46.183603 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.183563 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" Jul 22 20:12:46.184048 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.183604 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" event={"ID":"89db54bc-657c-4d1c-8f60-d8edef3dde39","Type":"ContainerDied","Data":"79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596"} Jul 22 20:12:46.184048 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.183662 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9" event={"ID":"89db54bc-657c-4d1c-8f60-d8edef3dde39","Type":"ContainerDied","Data":"3cf39c76ab3f21dfd02ba2de43b75b88c040f376c1f018c9bd41eb2d32e8a069"} Jul 22 20:12:46.184048 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.183690 2553 scope.go:117] "RemoveContainer" containerID="79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596" Jul 22 20:12:46.184048 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.183817 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:46.185341 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:46.185308 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="819deea3-aaec-475a-8c0b-e46a881aff37" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:46.191297 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:46.191210 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="819deea3-aaec-475a-8c0b-e46a881aff37" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:46.193545 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:46.193379 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="819deea3-aaec-475a-8c0b-e46a881aff37" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:46.194075 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.194060 2553 scope.go:117] "RemoveContainer" containerID="79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596" Jul 22 20:12:46.194446 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:46.194428 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596\": container with ID starting with 79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596 not found: ID does not exist" containerID="79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596" Jul 22 20:12:46.194500 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.194456 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596"} err="failed to get container status \"79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596\": rpc error: code = NotFound desc = could not find container \"79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596\": container with ID starting with 79b71ec616c7489d0397fca8da0a909ec38fd0cef8026e7baeaf7389abd98596 not found: ID does not exist" Jul 22 20:12:46.195786 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:12:46.195754 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nzm7z\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="819deea3-aaec-475a-8c0b-e46a881aff37" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nzm7z" Jul 22 20:12:46.209460 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.209431 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9"] Jul 22 20:12:46.215778 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:46.215748 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prrv9"] Jul 22 20:12:47.007270 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:47.007233 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="819deea3-aaec-475a-8c0b-e46a881aff37" path="/var/lib/kubelet/pods/819deea3-aaec-475a-8c0b-e46a881aff37/volumes" Jul 22 20:12:47.007525 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:47.007512 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89db54bc-657c-4d1c-8f60-d8edef3dde39" path="/var/lib/kubelet/pods/89db54bc-657c-4d1c-8f60-d8edef3dde39/volumes" Jul 22 20:12:54.173187 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:12:54.173154 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nlwsc" Jul 22 20:13:01.631387 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.631333 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt"] Jul 22 20:13:01.632090 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.632065 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a897897-2dbe-42d7-bd18-0c85f73ee0fc" containerName="console" Jul 22 20:13:01.632177 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.632093 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a897897-2dbe-42d7-bd18-0c85f73ee0fc" containerName="console" Jul 22 20:13:01.632177 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.632130 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89db54bc-657c-4d1c-8f60-d8edef3dde39" containerName="manager" Jul 22 20:13:01.632177 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.632139 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="89db54bc-657c-4d1c-8f60-d8edef3dde39" containerName="manager" Jul 22 20:13:01.632320 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.632261 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a897897-2dbe-42d7-bd18-0c85f73ee0fc" containerName="console" Jul 22 20:13:01.632320 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.632281 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="89db54bc-657c-4d1c-8f60-d8edef3dde39" containerName="manager" Jul 22 20:13:01.792528 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.792482 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt"] Jul 22 20:13:01.792697 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.792605 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:01.795015 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.794989 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-n5grn\"" Jul 22 20:13:01.795015 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.795007 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 20:13:01.918606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.918523 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:01.918606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.918599 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:01.918818 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.918623 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:01.918818 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.918641 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:01.918818 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.918672 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:01.918818 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.918760 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:01.918818 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.918807 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmk4d\" (UniqueName: \"kubernetes.io/projected/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-kube-api-access-jmk4d\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:01.919020 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.918871 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:01.919020 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:01.918904 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020280 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020232 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020280 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020279 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020556 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020301 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020556 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020337 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020556 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020383 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020556 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020430 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmk4d\" (UniqueName: \"kubernetes.io/projected/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-kube-api-access-jmk4d\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020556 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020467 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020556 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020496 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020851 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020564 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020851 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020765 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.020968 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.020949 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.021019 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.021000 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.021062 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.021030 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.021146 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.021125 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.022681 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.022660 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.022919 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.022900 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.028214 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.028183 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.028338 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.028319 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmk4d\" (UniqueName: \"kubernetes.io/projected/ae4f0dfa-c828-4b82-bd55-ea4ef01369b8-kube-api-access-jmk4d\") pod \"maas-default-gateway-openshift-default-56654dcf7-qjgnt\" (UID: \"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.108614 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.108566 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:02.304586 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:02.304558 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt"] Jul 22 20:13:02.306487 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:13:02.306461 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae4f0dfa_c828_4b82_bd55_ea4ef01369b8.slice/crio-d2cf876f82deaef81eb35c57f7be73068eaa73eb8e7ec5b4948fdf3084e5f8d3 WatchSource:0}: Error finding container d2cf876f82deaef81eb35c57f7be73068eaa73eb8e7ec5b4948fdf3084e5f8d3: Status 404 returned error can't find the container with id d2cf876f82deaef81eb35c57f7be73068eaa73eb8e7ec5b4948fdf3084e5f8d3 Jul 22 20:13:03.286052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:03.286015 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" event={"ID":"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8","Type":"ContainerStarted","Data":"d2cf876f82deaef81eb35c57f7be73068eaa73eb8e7ec5b4948fdf3084e5f8d3"} Jul 22 20:13:05.466127 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:05.466067 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 20:13:05.466458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:05.466166 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 20:13:05.466458 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:05.466200 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 20:13:06.317528 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:06.317486 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" event={"ID":"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8","Type":"ContainerStarted","Data":"087e9fbaffaa626ac0fdcf1188b1a1169f89f5025d5db912df492d2be9c977e3"} Jul 22 20:13:06.338487 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:06.338395 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" podStartSLOduration=2.181614898 podStartE2EDuration="5.338376861s" totalImagesPullingTime="3.156761963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:13:06.336588826 +0000 UTC m=+695.893332521" watchObservedRunningTime="2026-07-22 20:13:06.338376861 +0000 UTC m=+695.895120616" Jul 22 20:13:07.109829 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.109784 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:07.115076 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.115045 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:07.324354 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.324315 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:07.325499 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.325476 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:13:07.391645 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.391530 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h8ggm"] Jul 22 20:13:07.413788 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.413754 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h8ggm"] Jul 22 20:13:07.413992 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.413890 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:07.416120 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.416091 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 22 20:13:07.474222 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.474161 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fh4q\" (UniqueName: \"kubernetes.io/projected/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-kube-api-access-5fh4q\") pod \"limitador-limitador-64c94fd64f-h8ggm\" (UID: \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:07.474438 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.474250 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-config-file\") pod \"limitador-limitador-64c94fd64f-h8ggm\" (UID: \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:07.493083 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.493040 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h8ggm"] Jul 22 20:13:07.574907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.574861 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fh4q\" (UniqueName: \"kubernetes.io/projected/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-kube-api-access-5fh4q\") pod \"limitador-limitador-64c94fd64f-h8ggm\" (UID: \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:07.575112 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.574920 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-config-file\") pod \"limitador-limitador-64c94fd64f-h8ggm\" (UID: \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:07.575651 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.575625 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-config-file\") pod \"limitador-limitador-64c94fd64f-h8ggm\" (UID: \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:07.583492 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.583462 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fh4q\" (UniqueName: \"kubernetes.io/projected/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-kube-api-access-5fh4q\") pod \"limitador-limitador-64c94fd64f-h8ggm\" (UID: \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:07.727747 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.727716 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:07.881980 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:07.881948 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h8ggm"] Jul 22 20:13:07.883945 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:13:07.883906 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38dcbb6e_b5a8_4987_ad02_7e96d1ec5ca5.slice/crio-26b4ed76211470d935e55ed9e18e231a4f2b4242e3258c0699190ed5e9bbdf5b WatchSource:0}: Error finding container 26b4ed76211470d935e55ed9e18e231a4f2b4242e3258c0699190ed5e9bbdf5b: Status 404 returned error can't find the container with id 26b4ed76211470d935e55ed9e18e231a4f2b4242e3258c0699190ed5e9bbdf5b Jul 22 20:13:08.191715 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.191621 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-pbpm4"] Jul 22 20:13:08.196767 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.196740 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pbpm4" Jul 22 20:13:08.200323 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.200294 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9qwmd\"" Jul 22 20:13:08.200742 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.200723 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbpm4"] Jul 22 20:13:08.281792 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.281750 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d29kc\" (UniqueName: \"kubernetes.io/projected/5d577101-72e2-4dee-8330-64296155a0ef-kube-api-access-d29kc\") pod \"authorino-58cd955675-pbpm4\" (UID: \"5d577101-72e2-4dee-8330-64296155a0ef\") " pod="kuadrant-system/authorino-58cd955675-pbpm4" Jul 22 20:13:08.331706 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.331667 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" event={"ID":"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5","Type":"ContainerStarted","Data":"26b4ed76211470d935e55ed9e18e231a4f2b4242e3258c0699190ed5e9bbdf5b"} Jul 22 20:13:08.382669 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.382623 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d29kc\" (UniqueName: \"kubernetes.io/projected/5d577101-72e2-4dee-8330-64296155a0ef-kube-api-access-d29kc\") pod \"authorino-58cd955675-pbpm4\" (UID: \"5d577101-72e2-4dee-8330-64296155a0ef\") " pod="kuadrant-system/authorino-58cd955675-pbpm4" Jul 22 20:13:08.391591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.391557 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d29kc\" (UniqueName: \"kubernetes.io/projected/5d577101-72e2-4dee-8330-64296155a0ef-kube-api-access-d29kc\") pod \"authorino-58cd955675-pbpm4\" (UID: \"5d577101-72e2-4dee-8330-64296155a0ef\") " pod="kuadrant-system/authorino-58cd955675-pbpm4" Jul 22 20:13:08.508702 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.508656 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pbpm4" Jul 22 20:13:08.697542 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:08.697511 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbpm4"] Jul 22 20:13:08.698330 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:13:08.698300 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d577101_72e2_4dee_8330_64296155a0ef.slice/crio-431d1fdf5674a8c1167598829dd03ba8b313d9792ff6fdf193ff7bf35cff52e6 WatchSource:0}: Error finding container 431d1fdf5674a8c1167598829dd03ba8b313d9792ff6fdf193ff7bf35cff52e6: Status 404 returned error can't find the container with id 431d1fdf5674a8c1167598829dd03ba8b313d9792ff6fdf193ff7bf35cff52e6 Jul 22 20:13:09.340315 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:09.340256 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pbpm4" event={"ID":"5d577101-72e2-4dee-8330-64296155a0ef","Type":"ContainerStarted","Data":"431d1fdf5674a8c1167598829dd03ba8b313d9792ff6fdf193ff7bf35cff52e6"} Jul 22 20:13:11.358821 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:11.358780 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" event={"ID":"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5","Type":"ContainerStarted","Data":"a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936"} Jul 22 20:13:11.359235 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:11.358853 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:11.376116 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:11.376058 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" podStartSLOduration=1.5888817290000001 podStartE2EDuration="4.376041369s" totalImagesPullingTime="2.78715964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:13:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:13:11.374732078 +0000 UTC m=+700.931475804" watchObservedRunningTime="2026-07-22 20:13:11.376041369 +0000 UTC m=+700.932785097" Jul 22 20:13:12.444584 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:12.444537 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbpm4"] Jul 22 20:13:14.380466 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:14.380430 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pbpm4" event={"ID":"5d577101-72e2-4dee-8330-64296155a0ef","Type":"ContainerStarted","Data":"d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47"} Jul 22 20:13:14.380886 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:14.380472 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-pbpm4" podUID="5d577101-72e2-4dee-8330-64296155a0ef" containerName="authorino" containerID="cri-o://d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47" gracePeriod=30 Jul 22 20:13:14.394124 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:14.394068 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-pbpm4" podStartSLOduration=1.513637769 podStartE2EDuration="6.394049079s" totalImagesPullingTime="4.88041131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:13:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:13:14.393500347 +0000 UTC m=+703.950244042" watchObservedRunningTime="2026-07-22 20:13:14.394049079 +0000 UTC m=+703.950792775" Jul 22 20:13:14.637180 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:14.637121 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pbpm4" Jul 22 20:13:14.749767 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:14.749715 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d29kc\" (UniqueName: \"kubernetes.io/projected/5d577101-72e2-4dee-8330-64296155a0ef-kube-api-access-d29kc\") pod \"5d577101-72e2-4dee-8330-64296155a0ef\" (UID: \"5d577101-72e2-4dee-8330-64296155a0ef\") " Jul 22 20:13:14.751748 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:14.751722 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d577101-72e2-4dee-8330-64296155a0ef-kube-api-access-d29kc" (OuterVolumeSpecName: "kube-api-access-d29kc") pod "5d577101-72e2-4dee-8330-64296155a0ef" (UID: "5d577101-72e2-4dee-8330-64296155a0ef"). InnerVolumeSpecName "kube-api-access-d29kc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:13:14.850894 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:14.850839 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d29kc\" (UniqueName: \"kubernetes.io/projected/5d577101-72e2-4dee-8330-64296155a0ef-kube-api-access-d29kc\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:13:15.387915 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:15.387832 2553 generic.go:356] "Generic (PLEG): container finished" podID="5d577101-72e2-4dee-8330-64296155a0ef" containerID="d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47" exitCode=0 Jul 22 20:13:15.387915 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:15.387886 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pbpm4" Jul 22 20:13:15.388370 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:15.387918 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pbpm4" event={"ID":"5d577101-72e2-4dee-8330-64296155a0ef","Type":"ContainerDied","Data":"d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47"} Jul 22 20:13:15.388370 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:15.387964 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pbpm4" event={"ID":"5d577101-72e2-4dee-8330-64296155a0ef","Type":"ContainerDied","Data":"431d1fdf5674a8c1167598829dd03ba8b313d9792ff6fdf193ff7bf35cff52e6"} Jul 22 20:13:15.388370 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:15.387985 2553 scope.go:117] "RemoveContainer" containerID="d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47" Jul 22 20:13:15.396926 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:15.396909 2553 scope.go:117] "RemoveContainer" containerID="d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47" Jul 22 20:13:15.397183 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:13:15.397164 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47\": container with ID starting with d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47 not found: ID does not exist" containerID="d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47" Jul 22 20:13:15.397230 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:15.397192 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47"} err="failed to get container status \"d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47\": rpc error: code = NotFound desc = could not find container \"d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47\": container with ID starting with d45467b46ed40684d8b9b45b6640c7bf42e8115345b187b0ba3d19c4db273e47 not found: ID does not exist" Jul 22 20:13:15.403566 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:15.403546 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbpm4"] Jul 22 20:13:15.405837 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:15.405818 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-pbpm4"] Jul 22 20:13:17.007566 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:17.007534 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d577101-72e2-4dee-8330-64296155a0ef" path="/var/lib/kubelet/pods/5d577101-72e2-4dee-8330-64296155a0ef/volumes" Jul 22 20:13:22.367334 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:22.367305 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:22.384149 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:22.384109 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h8ggm"] Jul 22 20:13:22.433735 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:22.433691 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" podUID="38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5" containerName="limitador" containerID="cri-o://a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936" gracePeriod=30 Jul 22 20:13:22.996209 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:22.996144 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:23.125872 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.125837 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-config-file\") pod \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\" (UID: \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\") " Jul 22 20:13:23.126044 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.125913 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fh4q\" (UniqueName: \"kubernetes.io/projected/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-kube-api-access-5fh4q\") pod \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\" (UID: \"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5\") " Jul 22 20:13:23.126199 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.126177 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-config-file" (OuterVolumeSpecName: "config-file") pod "38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5" (UID: "38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:13:23.127927 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.127902 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-kube-api-access-5fh4q" (OuterVolumeSpecName: "kube-api-access-5fh4q") pod "38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5" (UID: "38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5"). InnerVolumeSpecName "kube-api-access-5fh4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:13:23.226817 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.226774 2553 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-config-file\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:13:23.226817 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.226807 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fh4q\" (UniqueName: \"kubernetes.io/projected/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5-kube-api-access-5fh4q\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:13:23.441390 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.441293 2553 generic.go:356] "Generic (PLEG): container finished" podID="38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5" containerID="a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936" exitCode=0 Jul 22 20:13:23.441390 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.441348 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" Jul 22 20:13:23.441390 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.441379 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" event={"ID":"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5","Type":"ContainerDied","Data":"a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936"} Jul 22 20:13:23.441963 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.441457 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h8ggm" event={"ID":"38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5","Type":"ContainerDied","Data":"26b4ed76211470d935e55ed9e18e231a4f2b4242e3258c0699190ed5e9bbdf5b"} Jul 22 20:13:23.441963 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.441481 2553 scope.go:117] "RemoveContainer" containerID="a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936" Jul 22 20:13:23.450889 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.450870 2553 scope.go:117] "RemoveContainer" containerID="a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936" Jul 22 20:13:23.451155 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:13:23.451134 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936\": container with ID starting with a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936 not found: ID does not exist" containerID="a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936" Jul 22 20:13:23.451206 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.451165 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936"} err="failed to get container status \"a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936\": rpc error: code = NotFound desc = could not find container \"a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936\": container with ID starting with a4a45a3911885f157167a9a0897f41fa4a145cf37bbbbe5637e81f07801f2936 not found: ID does not exist" Jul 22 20:13:23.462528 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.462502 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h8ggm"] Jul 22 20:13:23.466194 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:23.466170 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h8ggm"] Jul 22 20:13:24.166299 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.166266 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-z74nc"] Jul 22 20:13:24.166733 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.166718 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d577101-72e2-4dee-8330-64296155a0ef" containerName="authorino" Jul 22 20:13:24.166733 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.166733 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d577101-72e2-4dee-8330-64296155a0ef" containerName="authorino" Jul 22 20:13:24.166814 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.166750 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5" containerName="limitador" Jul 22 20:13:24.166814 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.166756 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5" containerName="limitador" Jul 22 20:13:24.166886 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.166816 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d577101-72e2-4dee-8330-64296155a0ef" containerName="authorino" Jul 22 20:13:24.166886 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.166829 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5" containerName="limitador" Jul 22 20:13:24.169513 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.169493 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:24.172588 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.172564 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 20:13:24.173222 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.173195 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cg2k8\"" Jul 22 20:13:24.173396 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.173301 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 20:13:24.173396 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.173319 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 22 20:13:24.182201 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.182180 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-z74nc"] Jul 22 20:13:24.339671 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.339638 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/80318c2e-2e37-4430-8e44-1715aea5e47a-data\") pod \"postgres-69fb4594cf-z74nc\" (UID: \"80318c2e-2e37-4430-8e44-1715aea5e47a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:24.339852 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.339712 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkz7k\" (UniqueName: \"kubernetes.io/projected/80318c2e-2e37-4430-8e44-1715aea5e47a-kube-api-access-rkz7k\") pod \"postgres-69fb4594cf-z74nc\" (UID: \"80318c2e-2e37-4430-8e44-1715aea5e47a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:24.440235 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.440137 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkz7k\" (UniqueName: \"kubernetes.io/projected/80318c2e-2e37-4430-8e44-1715aea5e47a-kube-api-access-rkz7k\") pod \"postgres-69fb4594cf-z74nc\" (UID: \"80318c2e-2e37-4430-8e44-1715aea5e47a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:24.440235 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.440227 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/80318c2e-2e37-4430-8e44-1715aea5e47a-data\") pod \"postgres-69fb4594cf-z74nc\" (UID: \"80318c2e-2e37-4430-8e44-1715aea5e47a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:24.440650 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.440634 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/80318c2e-2e37-4430-8e44-1715aea5e47a-data\") pod \"postgres-69fb4594cf-z74nc\" (UID: \"80318c2e-2e37-4430-8e44-1715aea5e47a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:24.448968 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.448937 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkz7k\" (UniqueName: \"kubernetes.io/projected/80318c2e-2e37-4430-8e44-1715aea5e47a-kube-api-access-rkz7k\") pod \"postgres-69fb4594cf-z74nc\" (UID: \"80318c2e-2e37-4430-8e44-1715aea5e47a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:24.480878 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.480846 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:24.624459 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:24.624397 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-z74nc"] Jul 22 20:13:24.625698 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:13:24.625672 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80318c2e_2e37_4430_8e44_1715aea5e47a.slice/crio-16aef9ff546b3327d4af60025425c7f0ae9f2ddc1069f49e5ed03e448a3ac0d7 WatchSource:0}: Error finding container 16aef9ff546b3327d4af60025425c7f0ae9f2ddc1069f49e5ed03e448a3ac0d7: Status 404 returned error can't find the container with id 16aef9ff546b3327d4af60025425c7f0ae9f2ddc1069f49e5ed03e448a3ac0d7 Jul 22 20:13:25.007521 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:25.007483 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5" path="/var/lib/kubelet/pods/38dcbb6e-b5a8-4987-ad02-7e96d1ec5ca5/volumes" Jul 22 20:13:25.455191 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:25.455110 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" event={"ID":"80318c2e-2e37-4430-8e44-1715aea5e47a","Type":"ContainerStarted","Data":"16aef9ff546b3327d4af60025425c7f0ae9f2ddc1069f49e5ed03e448a3ac0d7"} Jul 22 20:13:30.490907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:30.490866 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" event={"ID":"80318c2e-2e37-4430-8e44-1715aea5e47a","Type":"ContainerStarted","Data":"39e274d0a5ca1edc8e5ede8deb447421e8fef83cb84078dfb4153ea02ef77a40"} Jul 22 20:13:30.491304 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:30.490948 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:30.507452 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:30.507373 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" podStartSLOduration=1.046262382 podStartE2EDuration="6.507356661s" totalImagesPullingTime="5.461094279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:13:30.506233577 +0000 UTC m=+720.062977296" watchObservedRunningTime="2026-07-22 20:13:30.507356661 +0000 UTC m=+720.064100356" Jul 22 20:13:36.527854 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:36.527820 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z74nc" Jul 22 20:13:43.406695 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.406650 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5ff44bcff6-2fpfd"] Jul 22 20:13:43.414492 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.414459 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:43.421056 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.421026 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 20:13:43.425259 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.424945 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kkclf\"" Jul 22 20:13:43.429756 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.429554 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 20:13:43.433819 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.433781 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5ff44bcff6-2fpfd"] Jul 22 20:13:43.522586 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.522544 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m47zc\" (UniqueName: \"kubernetes.io/projected/9052a54b-8318-4d1d-87b7-dde6e4b8afac-kube-api-access-m47zc\") pod \"maas-controller-5ff44bcff6-2fpfd\" (UID: \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\") " pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:43.522764 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.522622 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9052a54b-8318-4d1d-87b7-dde6e4b8afac-webhook-cert\") pod \"maas-controller-5ff44bcff6-2fpfd\" (UID: \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\") " pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:43.585219 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.585179 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64cc9776b5-6tbz5"] Jul 22 20:13:43.588675 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.588651 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:43.598124 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.598089 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64cc9776b5-6tbz5"] Jul 22 20:13:43.623367 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.623330 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/026e2d00-dee6-4ef3-a12a-013d965b9de5-webhook-cert\") pod \"maas-controller-64cc9776b5-6tbz5\" (UID: \"026e2d00-dee6-4ef3-a12a-013d965b9de5\") " pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:43.623598 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.623381 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m47zc\" (UniqueName: \"kubernetes.io/projected/9052a54b-8318-4d1d-87b7-dde6e4b8afac-kube-api-access-m47zc\") pod \"maas-controller-5ff44bcff6-2fpfd\" (UID: \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\") " pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:43.623598 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.623469 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27rgc\" (UniqueName: \"kubernetes.io/projected/026e2d00-dee6-4ef3-a12a-013d965b9de5-kube-api-access-27rgc\") pod \"maas-controller-64cc9776b5-6tbz5\" (UID: \"026e2d00-dee6-4ef3-a12a-013d965b9de5\") " pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:43.623598 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.623580 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9052a54b-8318-4d1d-87b7-dde6e4b8afac-webhook-cert\") pod \"maas-controller-5ff44bcff6-2fpfd\" (UID: \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\") " pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:43.623788 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:13:43.623730 2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 22 20:13:43.623848 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:13:43.623801 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9052a54b-8318-4d1d-87b7-dde6e4b8afac-webhook-cert podName:9052a54b-8318-4d1d-87b7-dde6e4b8afac nodeName:}" failed. No retries permitted until 2026-07-22 20:13:44.123784521 +0000 UTC m=+733.680528194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9052a54b-8318-4d1d-87b7-dde6e4b8afac-webhook-cert") pod "maas-controller-5ff44bcff6-2fpfd" (UID: "9052a54b-8318-4d1d-87b7-dde6e4b8afac") : secret "maas-controller-webhook-cert" not found Jul 22 20:13:43.631989 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.631960 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m47zc\" (UniqueName: \"kubernetes.io/projected/9052a54b-8318-4d1d-87b7-dde6e4b8afac-kube-api-access-m47zc\") pod \"maas-controller-5ff44bcff6-2fpfd\" (UID: \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\") " pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:43.724681 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.724643 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/026e2d00-dee6-4ef3-a12a-013d965b9de5-webhook-cert\") pod \"maas-controller-64cc9776b5-6tbz5\" (UID: \"026e2d00-dee6-4ef3-a12a-013d965b9de5\") " pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:43.724851 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.724784 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27rgc\" (UniqueName: \"kubernetes.io/projected/026e2d00-dee6-4ef3-a12a-013d965b9de5-kube-api-access-27rgc\") pod \"maas-controller-64cc9776b5-6tbz5\" (UID: \"026e2d00-dee6-4ef3-a12a-013d965b9de5\") " pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:43.724894 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:13:43.724784 2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 22 20:13:43.724947 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:13:43.724934 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/026e2d00-dee6-4ef3-a12a-013d965b9de5-webhook-cert podName:026e2d00-dee6-4ef3-a12a-013d965b9de5 nodeName:}" failed. No retries permitted until 2026-07-22 20:13:44.224915183 +0000 UTC m=+733.781658859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/026e2d00-dee6-4ef3-a12a-013d965b9de5-webhook-cert") pod "maas-controller-64cc9776b5-6tbz5" (UID: "026e2d00-dee6-4ef3-a12a-013d965b9de5") : secret "maas-controller-webhook-cert" not found Jul 22 20:13:43.733521 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:43.733483 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27rgc\" (UniqueName: \"kubernetes.io/projected/026e2d00-dee6-4ef3-a12a-013d965b9de5-kube-api-access-27rgc\") pod \"maas-controller-64cc9776b5-6tbz5\" (UID: \"026e2d00-dee6-4ef3-a12a-013d965b9de5\") " pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:44.128883 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:44.128783 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9052a54b-8318-4d1d-87b7-dde6e4b8afac-webhook-cert\") pod \"maas-controller-5ff44bcff6-2fpfd\" (UID: \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\") " pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:44.131331 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:44.131299 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9052a54b-8318-4d1d-87b7-dde6e4b8afac-webhook-cert\") pod \"maas-controller-5ff44bcff6-2fpfd\" (UID: \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\") " pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:44.229496 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:44.229450 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/026e2d00-dee6-4ef3-a12a-013d965b9de5-webhook-cert\") pod \"maas-controller-64cc9776b5-6tbz5\" (UID: \"026e2d00-dee6-4ef3-a12a-013d965b9de5\") " pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:44.231977 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:44.231949 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/026e2d00-dee6-4ef3-a12a-013d965b9de5-webhook-cert\") pod \"maas-controller-64cc9776b5-6tbz5\" (UID: \"026e2d00-dee6-4ef3-a12a-013d965b9de5\") " pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:44.333848 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:44.333805 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:44.501329 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:44.501304 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:44.502548 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:44.502518 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5ff44bcff6-2fpfd"] Jul 22 20:13:44.504209 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:13:44.504183 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9052a54b_8318_4d1d_87b7_dde6e4b8afac.slice/crio-ebd6a1a6eadb70b699403a54f1c3cbca468a3381e741e64f3688c1521a7f5d00 WatchSource:0}: Error finding container ebd6a1a6eadb70b699403a54f1c3cbca468a3381e741e64f3688c1521a7f5d00: Status 404 returned error can't find the container with id ebd6a1a6eadb70b699403a54f1c3cbca468a3381e741e64f3688c1521a7f5d00 Jul 22 20:13:44.577732 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:44.577693 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" event={"ID":"9052a54b-8318-4d1d-87b7-dde6e4b8afac","Type":"ContainerStarted","Data":"ebd6a1a6eadb70b699403a54f1c3cbca468a3381e741e64f3688c1521a7f5d00"} Jul 22 20:13:44.651978 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:44.651940 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64cc9776b5-6tbz5"] Jul 22 20:13:44.653008 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:13:44.652981 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026e2d00_dee6_4ef3_a12a_013d965b9de5.slice/crio-f9bab5c1770e37ce0a2617fc9754e784118388ea944c5e015558caa3bf72b440 WatchSource:0}: Error finding container f9bab5c1770e37ce0a2617fc9754e784118388ea944c5e015558caa3bf72b440: Status 404 returned error can't find the container with id f9bab5c1770e37ce0a2617fc9754e784118388ea944c5e015558caa3bf72b440 Jul 22 20:13:45.591803 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:45.591717 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" event={"ID":"026e2d00-dee6-4ef3-a12a-013d965b9de5","Type":"ContainerStarted","Data":"f9bab5c1770e37ce0a2617fc9754e784118388ea944c5e015558caa3bf72b440"} Jul 22 20:13:47.608700 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:47.608651 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" event={"ID":"026e2d00-dee6-4ef3-a12a-013d965b9de5","Type":"ContainerStarted","Data":"0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286"} Jul 22 20:13:47.609162 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:47.608725 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:47.610292 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:47.610266 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" event={"ID":"9052a54b-8318-4d1d-87b7-dde6e4b8afac","Type":"ContainerStarted","Data":"9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976"} Jul 22 20:13:47.610437 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:47.610389 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:47.628973 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:47.628862 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" podStartSLOduration=1.8896416550000001 podStartE2EDuration="4.628844701s" totalImagesPullingTime="2.739203046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:13:47.6261294 +0000 UTC m=+737.182873095" watchObservedRunningTime="2026-07-22 20:13:47.628844701 +0000 UTC m=+737.185588430" Jul 22 20:13:47.655693 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:47.655632 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" podStartSLOduration=1.7789838169999999 podStartE2EDuration="4.655614707s" totalImagesPullingTime="2.87663089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:13:47.653020766 +0000 UTC m=+737.209764496" watchObservedRunningTime="2026-07-22 20:13:47.655614707 +0000 UTC m=+737.212358403" Jul 22 20:13:58.619234 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:58.619202 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:13:58.619634 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:58.619356 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:58.683502 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:58.683468 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5ff44bcff6-2fpfd"] Jul 22 20:13:58.687292 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:58.687252 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" podUID="9052a54b-8318-4d1d-87b7-dde6e4b8afac" containerName="manager" containerID="cri-o://9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976" gracePeriod=30 Jul 22 20:13:58.944489 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:58.944460 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:59.072046 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.072004 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m47zc\" (UniqueName: \"kubernetes.io/projected/9052a54b-8318-4d1d-87b7-dde6e4b8afac-kube-api-access-m47zc\") pod \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\" (UID: \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\") " Jul 22 20:13:59.072046 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.072048 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9052a54b-8318-4d1d-87b7-dde6e4b8afac-webhook-cert\") pod \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\" (UID: \"9052a54b-8318-4d1d-87b7-dde6e4b8afac\") " Jul 22 20:13:59.074297 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.074266 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9052a54b-8318-4d1d-87b7-dde6e4b8afac-kube-api-access-m47zc" (OuterVolumeSpecName: "kube-api-access-m47zc") pod "9052a54b-8318-4d1d-87b7-dde6e4b8afac" (UID: "9052a54b-8318-4d1d-87b7-dde6e4b8afac"). InnerVolumeSpecName "kube-api-access-m47zc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:13:59.074297 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.074288 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9052a54b-8318-4d1d-87b7-dde6e4b8afac-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9052a54b-8318-4d1d-87b7-dde6e4b8afac" (UID: "9052a54b-8318-4d1d-87b7-dde6e4b8afac"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:13:59.173278 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.173158 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m47zc\" (UniqueName: \"kubernetes.io/projected/9052a54b-8318-4d1d-87b7-dde6e4b8afac-kube-api-access-m47zc\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:13:59.173278 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.173215 2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9052a54b-8318-4d1d-87b7-dde6e4b8afac-webhook-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:13:59.694257 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.694222 2553 generic.go:356] "Generic (PLEG): container finished" podID="9052a54b-8318-4d1d-87b7-dde6e4b8afac" containerID="9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976" exitCode=0 Jul 22 20:13:59.694692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.694288 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" Jul 22 20:13:59.694692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.694305 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" event={"ID":"9052a54b-8318-4d1d-87b7-dde6e4b8afac","Type":"ContainerDied","Data":"9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976"} Jul 22 20:13:59.694692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.694391 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5ff44bcff6-2fpfd" event={"ID":"9052a54b-8318-4d1d-87b7-dde6e4b8afac","Type":"ContainerDied","Data":"ebd6a1a6eadb70b699403a54f1c3cbca468a3381e741e64f3688c1521a7f5d00"} Jul 22 20:13:59.694692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.694430 2553 scope.go:117] "RemoveContainer" containerID="9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976" Jul 22 20:13:59.704669 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.704652 2553 scope.go:117] "RemoveContainer" containerID="9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976" Jul 22 20:13:59.704939 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:13:59.704921 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976\": container with ID starting with 9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976 not found: ID does not exist" containerID="9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976" Jul 22 20:13:59.705003 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.704948 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976"} err="failed to get container status \"9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976\": rpc error: code = NotFound desc = could not find container \"9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976\": container with ID starting with 9dbc3b42899017aa7a284e61b4e665b9dade0af57440185be3be50a2ee94e976 not found: ID does not exist" Jul 22 20:13:59.715901 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.715877 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5ff44bcff6-2fpfd"] Jul 22 20:13:59.720063 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:13:59.720039 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5ff44bcff6-2fpfd"] Jul 22 20:14:01.008858 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:01.008820 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9052a54b-8318-4d1d-87b7-dde6e4b8afac" path="/var/lib/kubelet/pods/9052a54b-8318-4d1d-87b7-dde6e4b8afac/volumes" Jul 22 20:14:22.838074 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.838039 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26"] Jul 22 20:14:22.838991 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.838570 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9052a54b-8318-4d1d-87b7-dde6e4b8afac" containerName="manager" Jul 22 20:14:22.838991 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.838588 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9052a54b-8318-4d1d-87b7-dde6e4b8afac" containerName="manager" Jul 22 20:14:22.838991 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.838686 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9052a54b-8318-4d1d-87b7-dde6e4b8afac" containerName="manager" Jul 22 20:14:22.856289 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.856256 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26"] Jul 22 20:14:22.856457 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.856405 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:22.859080 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.859057 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mw4f6\"" Jul 22 20:14:22.860137 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.860118 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 22 20:14:22.896750 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.896722 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw78x\" (UniqueName: \"kubernetes.io/projected/0b17580e-b0cd-45a0-b536-07bcf3563ace-kube-api-access-jw78x\") pod \"maas-api-5d57d6b457-6fj26\" (UID: \"0b17580e-b0cd-45a0-b536-07bcf3563ace\") " pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:22.896937 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.896780 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b17580e-b0cd-45a0-b536-07bcf3563ace-openshift-service-ca\") pod \"maas-api-5d57d6b457-6fj26\" (UID: \"0b17580e-b0cd-45a0-b536-07bcf3563ace\") " pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:22.896937 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.896863 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b17580e-b0cd-45a0-b536-07bcf3563ace-maas-api-tls\") pod \"maas-api-5d57d6b457-6fj26\" (UID: \"0b17580e-b0cd-45a0-b536-07bcf3563ace\") " pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:22.997788 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.997757 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b17580e-b0cd-45a0-b536-07bcf3563ace-maas-api-tls\") pod \"maas-api-5d57d6b457-6fj26\" (UID: \"0b17580e-b0cd-45a0-b536-07bcf3563ace\") " pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:22.997956 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.997896 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw78x\" (UniqueName: \"kubernetes.io/projected/0b17580e-b0cd-45a0-b536-07bcf3563ace-kube-api-access-jw78x\") pod \"maas-api-5d57d6b457-6fj26\" (UID: \"0b17580e-b0cd-45a0-b536-07bcf3563ace\") " pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:22.997956 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.997946 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b17580e-b0cd-45a0-b536-07bcf3563ace-openshift-service-ca\") pod \"maas-api-5d57d6b457-6fj26\" (UID: \"0b17580e-b0cd-45a0-b536-07bcf3563ace\") " pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:22.998611 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:22.998587 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b17580e-b0cd-45a0-b536-07bcf3563ace-openshift-service-ca\") pod \"maas-api-5d57d6b457-6fj26\" (UID: \"0b17580e-b0cd-45a0-b536-07bcf3563ace\") " pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:23.000244 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.000224 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b17580e-b0cd-45a0-b536-07bcf3563ace-maas-api-tls\") pod \"maas-api-5d57d6b457-6fj26\" (UID: \"0b17580e-b0cd-45a0-b536-07bcf3563ace\") " pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:23.008101 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.008073 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw78x\" (UniqueName: \"kubernetes.io/projected/0b17580e-b0cd-45a0-b536-07bcf3563ace-kube-api-access-jw78x\") pod \"maas-api-5d57d6b457-6fj26\" (UID: \"0b17580e-b0cd-45a0-b536-07bcf3563ace\") " pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:23.168970 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.168879 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:23.328968 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:14:23.328830 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b17580e_b0cd_45a0_b536_07bcf3563ace.slice/crio-5048e7adf9c38c4487cbea5249a66958085e8df38d4fefe6d70a545487108980 WatchSource:0}: Error finding container 5048e7adf9c38c4487cbea5249a66958085e8df38d4fefe6d70a545487108980: Status 404 returned error can't find the container with id 5048e7adf9c38c4487cbea5249a66958085e8df38d4fefe6d70a545487108980 Jul 22 20:14:23.329381 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.329351 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26"] Jul 22 20:14:23.410596 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.410561 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-5qbzv"] Jul 22 20:14:23.415357 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.415336 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:23.419109 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.419051 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-g47jq"] Jul 22 20:14:23.420900 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.420881 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 22 20:14:23.422341 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.422322 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:23.453544 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.453474 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-5qbzv"] Jul 22 20:14:23.457517 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.457491 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-g47jq"] Jul 22 20:14:23.503126 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.503092 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvcf7\" (UniqueName: \"kubernetes.io/projected/2eb8d9e2-7b38-4512-8920-e1be5129916d-kube-api-access-zvcf7\") pod \"payload-processing-5bf47cd988-g47jq\" (UID: \"2eb8d9e2-7b38-4512-8920-e1be5129916d\") " pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:23.503301 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.503145 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdx2w\" (UniqueName: \"kubernetes.io/projected/072bf681-5686-470f-8e3f-f13659a49b9d-kube-api-access-mdx2w\") pod \"payload-pre-processing-7677b774bb-5qbzv\" (UID: \"072bf681-5686-470f-8e3f-f13659a49b9d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:23.503301 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.503280 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eb8d9e2-7b38-4512-8920-e1be5129916d-plugins-config-volume\") pod \"payload-processing-5bf47cd988-g47jq\" (UID: \"2eb8d9e2-7b38-4512-8920-e1be5129916d\") " pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:23.503403 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.503325 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/072bf681-5686-470f-8e3f-f13659a49b9d-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-5qbzv\" (UID: \"072bf681-5686-470f-8e3f-f13659a49b9d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:23.604739 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.604704 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eb8d9e2-7b38-4512-8920-e1be5129916d-plugins-config-volume\") pod \"payload-processing-5bf47cd988-g47jq\" (UID: \"2eb8d9e2-7b38-4512-8920-e1be5129916d\") " pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:23.604913 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.604752 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/072bf681-5686-470f-8e3f-f13659a49b9d-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-5qbzv\" (UID: \"072bf681-5686-470f-8e3f-f13659a49b9d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:23.604913 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.604795 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvcf7\" (UniqueName: \"kubernetes.io/projected/2eb8d9e2-7b38-4512-8920-e1be5129916d-kube-api-access-zvcf7\") pod \"payload-processing-5bf47cd988-g47jq\" (UID: \"2eb8d9e2-7b38-4512-8920-e1be5129916d\") " pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:23.604913 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.604821 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdx2w\" (UniqueName: \"kubernetes.io/projected/072bf681-5686-470f-8e3f-f13659a49b9d-kube-api-access-mdx2w\") pod \"payload-pre-processing-7677b774bb-5qbzv\" (UID: \"072bf681-5686-470f-8e3f-f13659a49b9d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:23.605734 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.605699 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/072bf681-5686-470f-8e3f-f13659a49b9d-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-5qbzv\" (UID: \"072bf681-5686-470f-8e3f-f13659a49b9d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:23.605911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.605699 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eb8d9e2-7b38-4512-8920-e1be5129916d-plugins-config-volume\") pod \"payload-processing-5bf47cd988-g47jq\" (UID: \"2eb8d9e2-7b38-4512-8920-e1be5129916d\") " pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:23.615039 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.615005 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvcf7\" (UniqueName: \"kubernetes.io/projected/2eb8d9e2-7b38-4512-8920-e1be5129916d-kube-api-access-zvcf7\") pod \"payload-processing-5bf47cd988-g47jq\" (UID: \"2eb8d9e2-7b38-4512-8920-e1be5129916d\") " pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:23.615279 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.615261 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdx2w\" (UniqueName: \"kubernetes.io/projected/072bf681-5686-470f-8e3f-f13659a49b9d-kube-api-access-mdx2w\") pod \"payload-pre-processing-7677b774bb-5qbzv\" (UID: \"072bf681-5686-470f-8e3f-f13659a49b9d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:23.727685 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.727643 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:23.733877 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.733852 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:23.841942 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.841848 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" event={"ID":"0b17580e-b0cd-45a0-b536-07bcf3563ace","Type":"ContainerStarted","Data":"5048e7adf9c38c4487cbea5249a66958085e8df38d4fefe6d70a545487108980"} Jul 22 20:14:23.893796 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:23.893765 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-5qbzv"] Jul 22 20:14:23.894138 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:14:23.894112 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072bf681_5686_470f_8e3f_f13659a49b9d.slice/crio-3b4ae7148968f5912b64a83830c1bbcd231b35144cb88652c594fe73eefa9fa3 WatchSource:0}: Error finding container 3b4ae7148968f5912b64a83830c1bbcd231b35144cb88652c594fe73eefa9fa3: Status 404 returned error can't find the container with id 3b4ae7148968f5912b64a83830c1bbcd231b35144cb88652c594fe73eefa9fa3 Jul 22 20:14:24.120500 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:24.120471 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-g47jq"] Jul 22 20:14:24.122891 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:14:24.122835 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eb8d9e2_7b38_4512_8920_e1be5129916d.slice/crio-1cfc58b966913e0157498e6125bc0bda58a457f9a1c6d2a4b70eae1f67188522 WatchSource:0}: Error finding container 1cfc58b966913e0157498e6125bc0bda58a457f9a1c6d2a4b70eae1f67188522: Status 404 returned error can't find the container with id 1cfc58b966913e0157498e6125bc0bda58a457f9a1c6d2a4b70eae1f67188522 Jul 22 20:14:24.856778 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:24.856660 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" event={"ID":"072bf681-5686-470f-8e3f-f13659a49b9d","Type":"ContainerStarted","Data":"3b4ae7148968f5912b64a83830c1bbcd231b35144cb88652c594fe73eefa9fa3"} Jul 22 20:14:24.858934 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:24.858853 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" event={"ID":"2eb8d9e2-7b38-4512-8920-e1be5129916d","Type":"ContainerStarted","Data":"1cfc58b966913e0157498e6125bc0bda58a457f9a1c6d2a4b70eae1f67188522"} Jul 22 20:14:26.898014 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:26.897920 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" event={"ID":"2eb8d9e2-7b38-4512-8920-e1be5129916d","Type":"ContainerStarted","Data":"926a99772459e186749cfa2252da3642abe7916e78c4f20fcf54628d03d8e132"} Jul 22 20:14:26.898573 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:26.898552 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:26.900169 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:26.900096 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" event={"ID":"072bf681-5686-470f-8e3f-f13659a49b9d","Type":"ContainerStarted","Data":"328bbd3d44674aca3a8932eb59af1f59438edf0a8fe180ba15c83bb09e927761"} Jul 22 20:14:26.900333 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:26.900318 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:26.902826 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:26.902566 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" event={"ID":"0b17580e-b0cd-45a0-b536-07bcf3563ace","Type":"ContainerStarted","Data":"aa25edbd36d71963ce9c5052a80aedc036c4752c9799f75225926299e765d84c"} Jul 22 20:14:26.902826 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:26.902641 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:26.919973 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:26.919909 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" podStartSLOduration=1.5301343520000001 podStartE2EDuration="3.919887399s" totalImagesPullingTime="2.389753047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:14:26.914016304 +0000 UTC m=+776.470759999" watchObservedRunningTime="2026-07-22 20:14:26.919887399 +0000 UTC m=+776.476631097" Jul 22 20:14:26.932878 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:26.932829 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" podStartSLOduration=1.306968947 podStartE2EDuration="3.932815465s" totalImagesPullingTime="2.625846518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:14:26.930167561 +0000 UTC m=+776.486911255" watchObservedRunningTime="2026-07-22 20:14:26.932815465 +0000 UTC m=+776.489559170" Jul 22 20:14:26.952629 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:26.952570 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" podStartSLOduration=2.4541811239999998 podStartE2EDuration="4.952550198s" totalImagesPullingTime="2.498369074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:14:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:14:26.948555554 +0000 UTC m=+776.505299267" watchObservedRunningTime="2026-07-22 20:14:26.952550198 +0000 UTC m=+776.509293894" Jul 22 20:14:31.940052 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:31.940017 2553 generic.go:356] "Generic (PLEG): container finished" podID="ae4f0dfa-c828-4b82-bd55-ea4ef01369b8" containerID="087e9fbaffaa626ac0fdcf1188b1a1169f89f5025d5db912df492d2be9c977e3" exitCode=0 Jul 22 20:14:31.940485 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:31.940091 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" event={"ID":"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8","Type":"ContainerDied","Data":"087e9fbaffaa626ac0fdcf1188b1a1169f89f5025d5db912df492d2be9c977e3"} Jul 22 20:14:31.940533 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:31.940511 2553 scope.go:117] "RemoveContainer" containerID="087e9fbaffaa626ac0fdcf1188b1a1169f89f5025d5db912df492d2be9c977e3" Jul 22 20:14:31.941583 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:31.941541 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 20:14:31.941673 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:31.941610 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 20:14:31.941673 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:31.941640 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 20:14:32.914955 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:32.914924 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5d57d6b457-6fj26" Jul 22 20:14:32.949082 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:32.949048 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" event={"ID":"ae4f0dfa-c828-4b82-bd55-ea4ef01369b8","Type":"ContainerStarted","Data":"62939fa1f4b19fc29866fed50b709e58a985dd93834c82bae22216a8f64c66c0"} Jul 22 20:14:33.109382 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:33.109345 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:14:33.110629 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:33.110606 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qjgnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 22 20:14:33.110697 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:33.110655 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" podUID="ae4f0dfa-c828-4b82-bd55-ea4ef01369b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 22 20:14:34.109391 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:34.109350 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qjgnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 22 20:14:34.109821 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:34.109438 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" podUID="ae4f0dfa-c828-4b82-bd55-ea4ef01369b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 22 20:14:35.110199 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:35.110151 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qjgnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 22 20:14:35.110724 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:35.110228 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" podUID="ae4f0dfa-c828-4b82-bd55-ea4ef01369b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 22 20:14:36.109912 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:36.109873 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qjgnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 22 20:14:36.110107 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:36.109929 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" podUID="ae4f0dfa-c828-4b82-bd55-ea4ef01369b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 22 20:14:37.113366 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:37.113325 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:14:37.113772 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:37.113668 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:14:37.114292 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:37.114271 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qjgnt" Jul 22 20:14:37.911798 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:37.911765 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-5qbzv" Jul 22 20:14:38.919469 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:38.919436 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-g47jq" Jul 22 20:14:45.528911 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.528864 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc"] Jul 22 20:14:45.532686 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.532659 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.535907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.535879 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 22 20:14:45.536079 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.535949 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tvrjw\"" Jul 22 20:14:45.536079 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.535958 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 20:14:45.536079 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.536017 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 20:14:45.544236 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.544203 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc"] Jul 22 20:14:45.639108 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.639066 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvwdw\" (UniqueName: \"kubernetes.io/projected/d849b31d-21d4-4b7e-859d-8ce6785a6e93-kube-api-access-tvwdw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.639302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.639126 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.639302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.639165 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d849b31d-21d4-4b7e-859d-8ce6785a6e93-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.639302 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.639191 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.639490 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.639313 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.639490 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.639358 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.740233 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.740184 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.740233 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.740235 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d849b31d-21d4-4b7e-859d-8ce6785a6e93-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.740490 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.740264 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.740490 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.740329 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.740490 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.740359 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.740490 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.740454 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvwdw\" (UniqueName: \"kubernetes.io/projected/d849b31d-21d4-4b7e-859d-8ce6785a6e93-kube-api-access-tvwdw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.740746 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.740722 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.740808 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.740768 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.740855 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.740818 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.742739 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.742703 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d849b31d-21d4-4b7e-859d-8ce6785a6e93-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.742885 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.742866 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d849b31d-21d4-4b7e-859d-8ce6785a6e93-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.748220 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.748190 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvwdw\" (UniqueName: \"kubernetes.io/projected/d849b31d-21d4-4b7e-859d-8ce6785a6e93-kube-api-access-tvwdw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc\" (UID: \"d849b31d-21d4-4b7e-859d-8ce6785a6e93\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.846816 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.846713 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:45.998196 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:45.998168 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc"] Jul 22 20:14:45.999868 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:14:45.999843 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd849b31d_21d4_4b7e_859d_8ce6785a6e93.slice/crio-c5b239dee75b8156eb37fbf150e10ba4bdd7fadd62f1d2a0ad3a061e4be4d601 WatchSource:0}: Error finding container c5b239dee75b8156eb37fbf150e10ba4bdd7fadd62f1d2a0ad3a061e4be4d601: Status 404 returned error can't find the container with id c5b239dee75b8156eb37fbf150e10ba4bdd7fadd62f1d2a0ad3a061e4be4d601 Jul 22 20:14:46.034217 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:46.034184 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" event={"ID":"d849b31d-21d4-4b7e-859d-8ce6785a6e93","Type":"ContainerStarted","Data":"c5b239dee75b8156eb37fbf150e10ba4bdd7fadd62f1d2a0ad3a061e4be4d601"} Jul 22 20:14:48.061029 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:48.060986 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" event={"ID":"d849b31d-21d4-4b7e-859d-8ce6785a6e93","Type":"ContainerStarted","Data":"05b11d851ca9b7bcac1983e7f98c48a6594c3e8e31845e8251838e4667434ae3"} Jul 22 20:14:48.079998 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:48.079940 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" podStartSLOduration=1.9306647049999999 podStartE2EDuration="3.079923698s" totalImagesPullingTime="1.149258993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:14:48.077221029 +0000 UTC m=+797.633964724" watchObservedRunningTime="2026-07-22 20:14:48.079923698 +0000 UTC m=+797.636667392" Jul 22 20:14:55.847171 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:55.847088 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:55.847171 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:55.847158 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:55.859524 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:55.859499 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:14:56.117593 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:14:56.117505 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc" Jul 22 20:15:00.143072 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:00.143035 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg"] Jul 22 20:15:00.147010 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:00.146986 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg" Jul 22 20:15:00.149242 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:00.149208 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hjqt7\"" Jul 22 20:15:00.153461 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:00.153437 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg"] Jul 22 20:15:00.159157 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:00.159136 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg" Jul 22 20:15:00.502662 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:00.502633 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg"] Jul 22 20:15:00.504368 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:15:00.504337 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7509fa_bf34_492d_b9e7_09a8cb3ddaf1.slice/crio-fe286ffaf0ec5eafa355126e0f90bd6c2a0b21f0d1c99a66289df02821af87c9 WatchSource:0}: Error finding container fe286ffaf0ec5eafa355126e0f90bd6c2a0b21f0d1c99a66289df02821af87c9: Status 404 returned error can't find the container with id fe286ffaf0ec5eafa355126e0f90bd6c2a0b21f0d1c99a66289df02821af87c9 Jul 22 20:15:01.135792 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:01.135757 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg" event={"ID":"3e7509fa-bf34-492d-b9e7-09a8cb3ddaf1","Type":"ContainerStarted","Data":"fe286ffaf0ec5eafa355126e0f90bd6c2a0b21f0d1c99a66289df02821af87c9"} Jul 22 20:15:04.157079 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:04.157039 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg" event={"ID":"3e7509fa-bf34-492d-b9e7-09a8cb3ddaf1","Type":"ContainerStarted","Data":"2b1de7f04c4d470c8f0c8beb94918e42019bfb78d9e5719290e9e0f1051fe316"} Jul 22 20:15:04.173018 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:04.172967 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg" podStartSLOduration=1.351301004 podStartE2EDuration="4.172951487s" totalImagesPullingTime="2.821650483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:15:04.17065484 +0000 UTC m=+813.727398536" watchObservedRunningTime="2026-07-22 20:15:04.172951487 +0000 UTC m=+813.729695225" Jul 22 20:15:05.164066 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:05.164033 2553 generic.go:356] "Generic (PLEG): container finished" podID="3e7509fa-bf34-492d-b9e7-09a8cb3ddaf1" containerID="2b1de7f04c4d470c8f0c8beb94918e42019bfb78d9e5719290e9e0f1051fe316" exitCode=0 Jul 22 20:15:05.164481 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:05.164120 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg" event={"ID":"3e7509fa-bf34-492d-b9e7-09a8cb3ddaf1","Type":"ContainerDied","Data":"2b1de7f04c4d470c8f0c8beb94918e42019bfb78d9e5719290e9e0f1051fe316"} Jul 22 20:15:06.314489 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:06.314461 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg" Jul 22 20:15:07.179784 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:07.179758 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg" Jul 22 20:15:07.179961 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:07.179801 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745855-72nrg" event={"ID":"3e7509fa-bf34-492d-b9e7-09a8cb3ddaf1","Type":"ContainerDied","Data":"fe286ffaf0ec5eafa355126e0f90bd6c2a0b21f0d1c99a66289df02821af87c9"} Jul 22 20:15:07.179961 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:07.179854 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe286ffaf0ec5eafa355126e0f90bd6c2a0b21f0d1c99a66289df02821af87c9" Jul 22 20:15:08.308757 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.308721 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5"] Jul 22 20:15:08.309246 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.309231 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e7509fa-bf34-492d-b9e7-09a8cb3ddaf1" containerName="cleanup" Jul 22 20:15:08.309246 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.309247 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e7509fa-bf34-492d-b9e7-09a8cb3ddaf1" containerName="cleanup" Jul 22 20:15:08.309359 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.309339 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e7509fa-bf34-492d-b9e7-09a8cb3ddaf1" containerName="cleanup" Jul 22 20:15:08.312641 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.312619 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.315031 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.315007 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 22 20:15:08.326245 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.326211 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5"] Jul 22 20:15:08.448271 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.448232 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.448466 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.448286 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c79304af-762e-48be-bc44-c1b46659e551-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.448466 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.448311 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.448466 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.448364 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.448466 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.448463 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnkjc\" (UniqueName: \"kubernetes.io/projected/c79304af-762e-48be-bc44-c1b46659e551-kube-api-access-bnkjc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.448737 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.448537 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.549242 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.549205 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.549469 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.549268 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c79304af-762e-48be-bc44-c1b46659e551-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.549469 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.549298 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.549469 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.549322 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.549469 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.549353 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnkjc\" (UniqueName: \"kubernetes.io/projected/c79304af-762e-48be-bc44-c1b46659e551-kube-api-access-bnkjc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.549720 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.549478 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.549787 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.549713 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.549849 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.549795 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.549849 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.549828 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.551644 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.551622 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c79304af-762e-48be-bc44-c1b46659e551-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.551891 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.551873 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c79304af-762e-48be-bc44-c1b46659e551-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.556823 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.556805 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnkjc\" (UniqueName: \"kubernetes.io/projected/c79304af-762e-48be-bc44-c1b46659e551-kube-api-access-bnkjc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7cjd5\" (UID: \"c79304af-762e-48be-bc44-c1b46659e551\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.626159 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.626069 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:08.769111 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:08.769076 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5"] Jul 22 20:15:08.770597 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:15:08.770565 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc79304af_762e_48be_bc44_c1b46659e551.slice/crio-d0eaa8aa84910c8536bc9dd91ee12d07f9828a5755823c125b6eeb2a51d5c897 WatchSource:0}: Error finding container d0eaa8aa84910c8536bc9dd91ee12d07f9828a5755823c125b6eeb2a51d5c897: Status 404 returned error can't find the container with id d0eaa8aa84910c8536bc9dd91ee12d07f9828a5755823c125b6eeb2a51d5c897 Jul 22 20:15:09.193823 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:09.193788 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" event={"ID":"c79304af-762e-48be-bc44-c1b46659e551","Type":"ContainerStarted","Data":"709cf359933db07e897a0d10cdfe4c0d3afeb1d7fb4e48fb27e4001c0be93900"} Jul 22 20:15:09.194000 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:09.193853 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" event={"ID":"c79304af-762e-48be-bc44-c1b46659e551","Type":"ContainerStarted","Data":"d0eaa8aa84910c8536bc9dd91ee12d07f9828a5755823c125b6eeb2a51d5c897"} Jul 22 20:15:09.220234 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:09.220175 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" podStartSLOduration=0.906949223 podStartE2EDuration="1.220158666s" totalImagesPullingTime="313.209443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:15:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:15:09.217167177 +0000 UTC m=+818.773910874" watchObservedRunningTime="2026-07-22 20:15:09.220158666 +0000 UTC m=+818.776902407" Jul 22 20:15:18.626824 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:18.626785 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:18.627244 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:18.626846 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:18.639455 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:18.639422 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:19.264343 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:19.264313 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7cjd5" Jul 22 20:15:24.228194 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.228151 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx"] Jul 22 20:15:24.232203 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.232183 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.234454 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.234404 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 22 20:15:24.240423 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.240380 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx"] Jul 22 20:15:24.402304 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.402258 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.402538 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.402321 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.402538 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.402387 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a9a876-bdf9-495d-9e31-839c948ab7a8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.402538 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.402455 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.402670 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.402555 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.402670 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.402578 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8v88\" (UniqueName: \"kubernetes.io/projected/77a9a876-bdf9-495d-9e31-839c948ab7a8-kube-api-access-h8v88\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.503966 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.503875 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.503966 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.503917 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8v88\" (UniqueName: \"kubernetes.io/projected/77a9a876-bdf9-495d-9e31-839c948ab7a8-kube-api-access-h8v88\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.504163 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.504037 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.504222 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.504163 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.504344 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.504320 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a9a876-bdf9-495d-9e31-839c948ab7a8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.504438 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.504380 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.504497 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.504400 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.504587 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.504564 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.504737 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.504715 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.506401 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.506375 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77a9a876-bdf9-495d-9e31-839c948ab7a8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.506807 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.506790 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a9a876-bdf9-495d-9e31-839c948ab7a8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.511294 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.511273 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8v88\" (UniqueName: \"kubernetes.io/projected/77a9a876-bdf9-495d-9e31-839c948ab7a8-kube-api-access-h8v88\") pod \"e2e-trlp-test-simulated-kserve-598466548d-clqhx\" (UID: \"77a9a876-bdf9-495d-9e31-839c948ab7a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.545091 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.545064 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:24.711816 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:24.711782 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx"] Jul 22 20:15:24.716846 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:15:24.716804 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a9a876_bdf9_495d_9e31_839c948ab7a8.slice/crio-d607246b5e8f9d1e67c5afad888263b544bdf424fc1df078a07554d9995b4f40 WatchSource:0}: Error finding container d607246b5e8f9d1e67c5afad888263b544bdf424fc1df078a07554d9995b4f40: Status 404 returned error can't find the container with id d607246b5e8f9d1e67c5afad888263b544bdf424fc1df078a07554d9995b4f40 Jul 22 20:15:25.302077 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:25.302039 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" event={"ID":"77a9a876-bdf9-495d-9e31-839c948ab7a8","Type":"ContainerStarted","Data":"c233c4b23596346fedac8f8f65e0de11d5e8a9625649a21da427f1d85c927b13"} Jul 22 20:15:25.302472 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:25.302126 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" event={"ID":"77a9a876-bdf9-495d-9e31-839c948ab7a8","Type":"ContainerStarted","Data":"d607246b5e8f9d1e67c5afad888263b544bdf424fc1df078a07554d9995b4f40"} Jul 22 20:15:25.319717 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:25.319669 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" podStartSLOduration=1.114257684 podStartE2EDuration="1.319653998s" totalImagesPullingTime="205.396314ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:15:25.318163629 +0000 UTC m=+834.874907359" watchObservedRunningTime="2026-07-22 20:15:25.319653998 +0000 UTC m=+834.876397693" Jul 22 20:15:34.545801 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:34.545751 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:34.546308 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:34.545821 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:34.559029 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:34.558995 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:15:35.375451 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:15:35.375406 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-clqhx" Jul 22 20:16:02.433456 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.433401 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5"] Jul 22 20:16:02.437723 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.437701 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.440308 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.440281 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 22 20:16:02.446970 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.446938 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5"] Jul 22 20:16:02.590568 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.590532 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.590776 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.590602 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.590776 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.590669 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.590776 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.590695 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.590776 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.590762 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.590971 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.590801 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdcbt\" (UniqueName: \"kubernetes.io/projected/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-kube-api-access-pdcbt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.691766 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.691653 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.691766 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.691713 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdcbt\" (UniqueName: \"kubernetes.io/projected/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-kube-api-access-pdcbt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.691984 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.691770 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.691984 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.691803 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.691984 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.691857 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.691984 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.691874 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.692211 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.692177 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.692314 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.692209 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.692314 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.692260 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.694292 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.694268 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.694551 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.694533 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.701766 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.701733 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdcbt\" (UniqueName: \"kubernetes.io/projected/91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3-kube-api-access-pdcbt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qdvf5\" (UID: \"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.749663 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.749603 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:02.916236 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:02.916198 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5"] Jul 22 20:16:02.918109 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:16:02.918061 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91dabbf3_80a8_4f6c_a2a8_e5c43f07d4f3.slice/crio-9c7e369e5ef793079244a936146530f1f4c41e947f2838718ae41b05bf50780e WatchSource:0}: Error finding container 9c7e369e5ef793079244a936146530f1f4c41e947f2838718ae41b05bf50780e: Status 404 returned error can't find the container with id 9c7e369e5ef793079244a936146530f1f4c41e947f2838718ae41b05bf50780e Jul 22 20:16:03.547851 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:03.547811 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" event={"ID":"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3","Type":"ContainerStarted","Data":"cde42f1b05e11e695fd61e15894aaaa24f759d3c77844fba399bdf9fc2cc44ed"} Jul 22 20:16:03.548289 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:03.547870 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" event={"ID":"91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3","Type":"ContainerStarted","Data":"9c7e369e5ef793079244a936146530f1f4c41e947f2838718ae41b05bf50780e"} Jul 22 20:16:03.567281 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:03.567141 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" podStartSLOduration=1.18137399 podStartE2EDuration="1.567118832s" totalImagesPullingTime="385.744842ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:16:03.563813814 +0000 UTC m=+873.120557510" watchObservedRunningTime="2026-07-22 20:16:03.567118832 +0000 UTC m=+873.123862527" Jul 22 20:16:12.750183 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:12.750128 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:12.750740 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:12.750230 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:12.763611 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:12.763580 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:13.625886 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:13.625858 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qdvf5" Jul 22 20:16:30.954819 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:30.954724 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:16:30.957870 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:16:30.957839 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:18:21.833433 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:21.833365 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64cc9776b5-6tbz5"] Jul 22 20:18:21.834030 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:21.833632 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" podUID="026e2d00-dee6-4ef3-a12a-013d965b9de5" containerName="manager" containerID="cri-o://0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286" gracePeriod=30 Jul 22 20:18:22.107335 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.107300 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:18:22.153142 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.153051 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/026e2d00-dee6-4ef3-a12a-013d965b9de5-webhook-cert\") pod \"026e2d00-dee6-4ef3-a12a-013d965b9de5\" (UID: \"026e2d00-dee6-4ef3-a12a-013d965b9de5\") " Jul 22 20:18:22.153592 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.153324 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27rgc\" (UniqueName: \"kubernetes.io/projected/026e2d00-dee6-4ef3-a12a-013d965b9de5-kube-api-access-27rgc\") pod \"026e2d00-dee6-4ef3-a12a-013d965b9de5\" (UID: \"026e2d00-dee6-4ef3-a12a-013d965b9de5\") " Jul 22 20:18:22.157159 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.157074 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/026e2d00-dee6-4ef3-a12a-013d965b9de5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "026e2d00-dee6-4ef3-a12a-013d965b9de5" (UID: "026e2d00-dee6-4ef3-a12a-013d965b9de5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:18:22.157559 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.157144 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026e2d00-dee6-4ef3-a12a-013d965b9de5-kube-api-access-27rgc" (OuterVolumeSpecName: "kube-api-access-27rgc") pod "026e2d00-dee6-4ef3-a12a-013d965b9de5" (UID: "026e2d00-dee6-4ef3-a12a-013d965b9de5"). InnerVolumeSpecName "kube-api-access-27rgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:18:22.254781 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.254728 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27rgc\" (UniqueName: \"kubernetes.io/projected/026e2d00-dee6-4ef3-a12a-013d965b9de5-kube-api-access-27rgc\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:18:22.254781 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.254769 2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/026e2d00-dee6-4ef3-a12a-013d965b9de5-webhook-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:18:22.439087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.438984 2553 generic.go:356] "Generic (PLEG): container finished" podID="026e2d00-dee6-4ef3-a12a-013d965b9de5" containerID="0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286" exitCode=0 Jul 22 20:18:22.439087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.439039 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" Jul 22 20:18:22.439087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.439065 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" event={"ID":"026e2d00-dee6-4ef3-a12a-013d965b9de5","Type":"ContainerDied","Data":"0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286"} Jul 22 20:18:22.439476 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.439120 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64cc9776b5-6tbz5" event={"ID":"026e2d00-dee6-4ef3-a12a-013d965b9de5","Type":"ContainerDied","Data":"f9bab5c1770e37ce0a2617fc9754e784118388ea944c5e015558caa3bf72b440"} Jul 22 20:18:22.439476 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.439145 2553 scope.go:117] "RemoveContainer" containerID="0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286" Jul 22 20:18:22.449993 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.449969 2553 scope.go:117] "RemoveContainer" containerID="0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286" Jul 22 20:18:22.450375 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:18:22.450350 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286\": container with ID starting with 0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286 not found: ID does not exist" containerID="0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286" Jul 22 20:18:22.450465 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.450388 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286"} err="failed to get container status \"0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286\": rpc error: code = NotFound desc = could not find container \"0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286\": container with ID starting with 0449a6047de78d6019b24276c26c2bfb8e613ffc996d21523b6af18b69f26286 not found: ID does not exist" Jul 22 20:18:22.462949 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.462908 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64cc9776b5-6tbz5"] Jul 22 20:18:22.467058 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:22.467016 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64cc9776b5-6tbz5"] Jul 22 20:18:23.009974 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:18:23.009935 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="026e2d00-dee6-4ef3-a12a-013d965b9de5" path="/var/lib/kubelet/pods/026e2d00-dee6-4ef3-a12a-013d965b9de5/volumes" Jul 22 20:21:31.002483 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:21:31.002371 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:21:31.008811 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:21:31.008777 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:26:31.050049 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:26:31.050012 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:26:31.057914 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:26:31.057879 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:31:31.095523 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:31:31.095490 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:31:31.107048 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:31:31.107021 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:32:40.361190 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.361152 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r"] Jul 22 20:32:40.361902 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.361882 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026e2d00-dee6-4ef3-a12a-013d965b9de5" containerName="manager" Jul 22 20:32:40.361994 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.361904 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="026e2d00-dee6-4ef3-a12a-013d965b9de5" containerName="manager" Jul 22 20:32:40.362056 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.362043 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="026e2d00-dee6-4ef3-a12a-013d965b9de5" containerName="manager" Jul 22 20:32:40.365284 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.365261 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:40.367706 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.367680 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e8c31dad-serving-cert\"" Jul 22 20:32:40.375824 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.375640 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r"] Jul 22 20:32:40.451507 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.451464 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd51b705-ffd9-4613-8773-f06e593e90a6-openshift-service-ca\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:40.451691 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.451525 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbzfl\" (UniqueName: \"kubernetes.io/projected/fd51b705-ffd9-4613-8773-f06e593e90a6-kube-api-access-fbzfl\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:40.451691 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.451570 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd51b705-ffd9-4613-8773-f06e593e90a6-maas-api-tls\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:40.552551 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.552509 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd51b705-ffd9-4613-8773-f06e593e90a6-maas-api-tls\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:40.552765 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.552621 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd51b705-ffd9-4613-8773-f06e593e90a6-openshift-service-ca\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:40.552765 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.552673 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbzfl\" (UniqueName: \"kubernetes.io/projected/fd51b705-ffd9-4613-8773-f06e593e90a6-kube-api-access-fbzfl\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:40.552765 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:32:40.552673 2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-serving-cert: secret "maas-api-e2e-ait-e8c31dad-serving-cert" not found Jul 22 20:32:40.552931 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:32:40.552785 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd51b705-ffd9-4613-8773-f06e593e90a6-maas-api-tls podName:fd51b705-ffd9-4613-8773-f06e593e90a6 nodeName:}" failed. No retries permitted until 2026-07-22 20:32:41.05276591 +0000 UTC m=+1870.609509586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fd51b705-ffd9-4613-8773-f06e593e90a6-maas-api-tls") pod "maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" (UID: "fd51b705-ffd9-4613-8773-f06e593e90a6") : secret "maas-api-e2e-ait-e8c31dad-serving-cert" not found Jul 22 20:32:40.553293 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.553272 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd51b705-ffd9-4613-8773-f06e593e90a6-openshift-service-ca\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:40.563994 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:40.563967 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbzfl\" (UniqueName: \"kubernetes.io/projected/fd51b705-ffd9-4613-8773-f06e593e90a6-kube-api-access-fbzfl\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:41.056883 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:41.056850 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd51b705-ffd9-4613-8773-f06e593e90a6-maas-api-tls\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:41.059323 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:41.059299 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd51b705-ffd9-4613-8773-f06e593e90a6-maas-api-tls\") pod \"maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:41.279725 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:41.279690 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:41.427517 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:41.427487 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r"] Jul 22 20:32:41.429367 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:32:41.429341 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd51b705_ffd9_4613_8773_f06e593e90a6.slice/crio-091da065e2c14dbbc03562b69b163c960ba4647856c0b0059b3374aa5a59f303 WatchSource:0}: Error finding container 091da065e2c14dbbc03562b69b163c960ba4647856c0b0059b3374aa5a59f303: Status 404 returned error can't find the container with id 091da065e2c14dbbc03562b69b163c960ba4647856c0b0059b3374aa5a59f303 Jul 22 20:32:41.430779 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:41.430762 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 20:32:41.958459 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:41.958425 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" event={"ID":"fd51b705-ffd9-4613-8773-f06e593e90a6","Type":"ContainerStarted","Data":"091da065e2c14dbbc03562b69b163c960ba4647856c0b0059b3374aa5a59f303"} Jul 22 20:32:44.978740 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:44.978700 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" event={"ID":"fd51b705-ffd9-4613-8773-f06e593e90a6","Type":"ContainerStarted","Data":"91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909"} Jul 22 20:32:44.979172 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:44.978764 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:32:44.995692 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:44.995638 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" podStartSLOduration=1.777788733 podStartE2EDuration="4.995621371s" totalImagesPullingTime="3.217832638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:32:44.993687405 +0000 UTC m=+1874.550431101" watchObservedRunningTime="2026-07-22 20:32:44.995621371 +0000 UTC m=+1874.552365087" Jul 22 20:32:50.995463 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:32:50.991759 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:33:16.349956 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.349843 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r"] Jul 22 20:33:16.350577 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.350259 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" podUID="fd51b705-ffd9-4613-8773-f06e593e90a6" containerName="maas-api" containerID="cri-o://91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909" gracePeriod=30 Jul 22 20:33:16.612581 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.612517 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:33:16.724144 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.724105 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbzfl\" (UniqueName: \"kubernetes.io/projected/fd51b705-ffd9-4613-8773-f06e593e90a6-kube-api-access-fbzfl\") pod \"fd51b705-ffd9-4613-8773-f06e593e90a6\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " Jul 22 20:33:16.724355 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.724186 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd51b705-ffd9-4613-8773-f06e593e90a6-openshift-service-ca\") pod \"fd51b705-ffd9-4613-8773-f06e593e90a6\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " Jul 22 20:33:16.724467 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.724401 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd51b705-ffd9-4613-8773-f06e593e90a6-maas-api-tls\") pod \"fd51b705-ffd9-4613-8773-f06e593e90a6\" (UID: \"fd51b705-ffd9-4613-8773-f06e593e90a6\") " Jul 22 20:33:16.724621 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.724599 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd51b705-ffd9-4613-8773-f06e593e90a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd51b705-ffd9-4613-8773-f06e593e90a6" (UID: "fd51b705-ffd9-4613-8773-f06e593e90a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:33:16.724855 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.724837 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd51b705-ffd9-4613-8773-f06e593e90a6-openshift-service-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:33:16.726656 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.726624 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd51b705-ffd9-4613-8773-f06e593e90a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd51b705-ffd9-4613-8773-f06e593e90a6" (UID: "fd51b705-ffd9-4613-8773-f06e593e90a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:33:16.726748 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.726673 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd51b705-ffd9-4613-8773-f06e593e90a6-kube-api-access-fbzfl" (OuterVolumeSpecName: "kube-api-access-fbzfl") pod "fd51b705-ffd9-4613-8773-f06e593e90a6" (UID: "fd51b705-ffd9-4613-8773-f06e593e90a6"). InnerVolumeSpecName "kube-api-access-fbzfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:33:16.825709 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.825672 2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd51b705-ffd9-4613-8773-f06e593e90a6-maas-api-tls\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:33:16.825709 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:16.825701 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbzfl\" (UniqueName: \"kubernetes.io/projected/fd51b705-ffd9-4613-8773-f06e593e90a6-kube-api-access-fbzfl\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:33:17.179900 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:17.179865 2553 generic.go:356] "Generic (PLEG): container finished" podID="fd51b705-ffd9-4613-8773-f06e593e90a6" containerID="91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909" exitCode=0 Jul 22 20:33:17.180087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:17.179914 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" Jul 22 20:33:17.180087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:17.179947 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" event={"ID":"fd51b705-ffd9-4613-8773-f06e593e90a6","Type":"ContainerDied","Data":"91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909"} Jul 22 20:33:17.180087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:17.179997 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r" event={"ID":"fd51b705-ffd9-4613-8773-f06e593e90a6","Type":"ContainerDied","Data":"091da065e2c14dbbc03562b69b163c960ba4647856c0b0059b3374aa5a59f303"} Jul 22 20:33:17.180087 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:17.180019 2553 scope.go:117] "RemoveContainer" containerID="91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909" Jul 22 20:33:17.190247 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:17.190223 2553 scope.go:117] "RemoveContainer" containerID="91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909" Jul 22 20:33:17.190620 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:33:17.190603 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909\": container with ID starting with 91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909 not found: ID does not exist" containerID="91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909" Jul 22 20:33:17.190688 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:17.190634 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909"} err="failed to get container status \"91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909\": rpc error: code = NotFound desc = could not find container \"91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909\": container with ID starting with 91e5b75c1ecb439904eaaef142b54efdb439fb23693c8292473131a768bc1909 not found: ID does not exist" Jul 22 20:33:17.198242 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:17.198210 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r"] Jul 22 20:33:17.201085 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:17.201057 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8c31dad-7c5f44f777-9787r"] Jul 22 20:33:19.007807 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:19.007764 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd51b705-ffd9-4613-8773-f06e593e90a6" path="/var/lib/kubelet/pods/fd51b705-ffd9-4613-8773-f06e593e90a6/volumes" Jul 22 20:33:24.586308 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.586269 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd"] Jul 22 20:33:24.587028 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.587012 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd51b705-ffd9-4613-8773-f06e593e90a6" containerName="maas-api" Jul 22 20:33:24.587082 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.587032 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd51b705-ffd9-4613-8773-f06e593e90a6" containerName="maas-api" Jul 22 20:33:24.587153 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.587142 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd51b705-ffd9-4613-8773-f06e593e90a6" containerName="maas-api" Jul 22 20:33:24.592265 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.592239 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:24.595266 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.595231 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-cef58190-serving-cert\"" Jul 22 20:33:24.598822 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.598794 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd"] Jul 22 20:33:24.703647 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.703612 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlhpk\" (UniqueName: \"kubernetes.io/projected/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-kube-api-access-zlhpk\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:24.703870 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.703667 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-openshift-service-ca\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:24.703870 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.703779 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-maas-api-tls\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:24.804317 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.804282 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlhpk\" (UniqueName: \"kubernetes.io/projected/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-kube-api-access-zlhpk\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:24.804317 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.804330 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-openshift-service-ca\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:24.804613 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.804381 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-maas-api-tls\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:24.804613 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:33:24.804559 2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-serving-cert: secret "maas-api-e2e-migrate-cef58190-serving-cert" not found Jul 22 20:33:24.804723 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:33:24.804645 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-maas-api-tls podName:98a5c11c-fc0a-4963-b1b8-25b14c2bca72 nodeName:}" failed. No retries permitted until 2026-07-22 20:33:25.304621972 +0000 UTC m=+1914.861365650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-maas-api-tls") pod "maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" (UID: "98a5c11c-fc0a-4963-b1b8-25b14c2bca72") : secret "maas-api-e2e-migrate-cef58190-serving-cert" not found Jul 22 20:33:24.805150 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.805124 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-openshift-service-ca\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:24.814027 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:24.813991 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlhpk\" (UniqueName: \"kubernetes.io/projected/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-kube-api-access-zlhpk\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:25.309676 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:25.309640 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-maas-api-tls\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:25.312375 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:25.312346 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-maas-api-tls\") pod \"maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:25.506870 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:25.506813 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:25.670060 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:25.670027 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd"] Jul 22 20:33:25.672125 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:33:25.672075 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a5c11c_fc0a_4963_b1b8_25b14c2bca72.slice/crio-0741eeb8cf99f078ed40c93a483b2501100880b727606f5b58917cd0215e5be8 WatchSource:0}: Error finding container 0741eeb8cf99f078ed40c93a483b2501100880b727606f5b58917cd0215e5be8: Status 404 returned error can't find the container with id 0741eeb8cf99f078ed40c93a483b2501100880b727606f5b58917cd0215e5be8 Jul 22 20:33:26.237170 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:26.237123 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" event={"ID":"98a5c11c-fc0a-4963-b1b8-25b14c2bca72","Type":"ContainerStarted","Data":"a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2"} Jul 22 20:33:26.237341 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:26.237220 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" event={"ID":"98a5c11c-fc0a-4963-b1b8-25b14c2bca72","Type":"ContainerStarted","Data":"0741eeb8cf99f078ed40c93a483b2501100880b727606f5b58917cd0215e5be8"} Jul 22 20:33:26.237341 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:26.237269 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:26.255165 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:26.255100 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" podStartSLOduration=1.898002856 podStartE2EDuration="2.25508499s" totalImagesPullingTime="357.082134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:33:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:33:26.252759659 +0000 UTC m=+1915.809503355" watchObservedRunningTime="2026-07-22 20:33:26.25508499 +0000 UTC m=+1915.811828741" Jul 22 20:33:32.248352 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:32.248323 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:33:59.891158 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:59.891077 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd"] Jul 22 20:33:59.892093 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:33:59.891477 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" podUID="98a5c11c-fc0a-4963-b1b8-25b14c2bca72" containerName="maas-api" containerID="cri-o://a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2" gracePeriod=30 Jul 22 20:34:00.159827 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.159750 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:34:00.251660 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.251612 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-openshift-service-ca\") pod \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " Jul 22 20:34:00.251660 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.251661 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-maas-api-tls\") pod \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " Jul 22 20:34:00.251883 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.251697 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlhpk\" (UniqueName: \"kubernetes.io/projected/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-kube-api-access-zlhpk\") pod \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\" (UID: \"98a5c11c-fc0a-4963-b1b8-25b14c2bca72\") " Jul 22 20:34:00.252102 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.252081 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98a5c11c-fc0a-4963-b1b8-25b14c2bca72" (UID: "98a5c11c-fc0a-4963-b1b8-25b14c2bca72"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:34:00.254127 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.254090 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-kube-api-access-zlhpk" (OuterVolumeSpecName: "kube-api-access-zlhpk") pod "98a5c11c-fc0a-4963-b1b8-25b14c2bca72" (UID: "98a5c11c-fc0a-4963-b1b8-25b14c2bca72"). InnerVolumeSpecName "kube-api-access-zlhpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:34:00.254127 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.254110 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98a5c11c-fc0a-4963-b1b8-25b14c2bca72" (UID: "98a5c11c-fc0a-4963-b1b8-25b14c2bca72"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 20:34:00.352897 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.352857 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-openshift-service-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:34:00.352897 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.352893 2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-maas-api-tls\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:34:00.352897 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.352903 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlhpk\" (UniqueName: \"kubernetes.io/projected/98a5c11c-fc0a-4963-b1b8-25b14c2bca72-kube-api-access-zlhpk\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:34:00.456361 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.456238 2553 generic.go:356] "Generic (PLEG): container finished" podID="98a5c11c-fc0a-4963-b1b8-25b14c2bca72" containerID="a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2" exitCode=0 Jul 22 20:34:00.456361 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.456307 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" Jul 22 20:34:00.456361 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.456330 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" event={"ID":"98a5c11c-fc0a-4963-b1b8-25b14c2bca72","Type":"ContainerDied","Data":"a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2"} Jul 22 20:34:00.456673 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.456404 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd" event={"ID":"98a5c11c-fc0a-4963-b1b8-25b14c2bca72","Type":"ContainerDied","Data":"0741eeb8cf99f078ed40c93a483b2501100880b727606f5b58917cd0215e5be8"} Jul 22 20:34:00.456673 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.456488 2553 scope.go:117] "RemoveContainer" containerID="a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2" Jul 22 20:34:00.470298 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.470263 2553 scope.go:117] "RemoveContainer" containerID="a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2" Jul 22 20:34:00.470739 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:34:00.470710 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2\": container with ID starting with a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2 not found: ID does not exist" containerID="a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2" Jul 22 20:34:00.470912 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.470747 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2"} err="failed to get container status \"a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2\": rpc error: code = NotFound desc = could not find container \"a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2\": container with ID starting with a148de62f1d5ac318c56fb18b6c8d4354d5b023c40c07606fa63f81cb140d7f2 not found: ID does not exist" Jul 22 20:34:00.483903 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.483858 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd"] Jul 22 20:34:00.487109 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:00.487068 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cef58190-77f7b49558-dfbdd"] Jul 22 20:34:01.008147 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:34:01.008104 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98a5c11c-fc0a-4963-b1b8-25b14c2bca72" path="/var/lib/kubelet/pods/98a5c11c-fc0a-4963-b1b8-25b14c2bca72/volumes" Jul 22 20:36:31.139589 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:36:31.139559 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:36:31.153199 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:36:31.153174 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:38:42.273703 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.273663 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l"] Jul 22 20:38:42.274394 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.274373 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a5c11c-fc0a-4963-b1b8-25b14c2bca72" containerName="maas-api" Jul 22 20:38:42.274540 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.274397 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a5c11c-fc0a-4963-b1b8-25b14c2bca72" containerName="maas-api" Jul 22 20:38:42.274540 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.274534 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a5c11c-fc0a-4963-b1b8-25b14c2bca72" containerName="maas-api" Jul 22 20:38:42.277702 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.277681 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" Jul 22 20:38:42.280308 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.280179 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hjqt7\"" Jul 22 20:38:42.284849 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.284825 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l"] Jul 22 20:38:42.369300 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.369269 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7edb7b62-210d-463c-956c-028b89c3f28f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e950683d-4vp6l\" (UID: \"7edb7b62-210d-463c-956c-028b89c3f28f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" Jul 22 20:38:42.470691 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.470656 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7edb7b62-210d-463c-956c-028b89c3f28f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e950683d-4vp6l\" (UID: \"7edb7b62-210d-463c-956c-028b89c3f28f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" Jul 22 20:38:42.471300 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.471279 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7edb7b62-210d-463c-956c-028b89c3f28f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e950683d-4vp6l\" (UID: \"7edb7b62-210d-463c-956c-028b89c3f28f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" Jul 22 20:38:42.589754 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.589669 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" Jul 22 20:38:42.728655 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:42.728630 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l"] Jul 22 20:38:42.729951 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:38:42.729915 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7edb7b62_210d_463c_956c_028b89c3f28f.slice/crio-e0498be8f8ee414b052a66aeea782567e1bbfff7c73a5342f9da68fe0966502f WatchSource:0}: Error finding container e0498be8f8ee414b052a66aeea782567e1bbfff7c73a5342f9da68fe0966502f: Status 404 returned error can't find the container with id e0498be8f8ee414b052a66aeea782567e1bbfff7c73a5342f9da68fe0966502f Jul 22 20:38:43.224605 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:43.224569 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" event={"ID":"7edb7b62-210d-463c-956c-028b89c3f28f","Type":"ContainerStarted","Data":"feb6caaa0f33a1208f010f5028985540de3a04d323e044a070682bbaee919f6d"} Jul 22 20:38:43.224763 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:43.224634 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" event={"ID":"7edb7b62-210d-463c-956c-028b89c3f28f","Type":"ContainerStarted","Data":"e0498be8f8ee414b052a66aeea782567e1bbfff7c73a5342f9da68fe0966502f"} Jul 22 20:38:43.242401 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:43.242354 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" podStartSLOduration=1.242341298 podStartE2EDuration="1.242341298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:38:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:38:43.239965204 +0000 UTC m=+2232.796708918" watchObservedRunningTime="2026-07-22 20:38:43.242341298 +0000 UTC m=+2232.799084993" Jul 22 20:38:44.231517 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:44.231475 2553 generic.go:356] "Generic (PLEG): container finished" podID="7edb7b62-210d-463c-956c-028b89c3f28f" containerID="feb6caaa0f33a1208f010f5028985540de3a04d323e044a070682bbaee919f6d" exitCode=0 Jul 22 20:38:44.231904 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:44.231567 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" event={"ID":"7edb7b62-210d-463c-956c-028b89c3f28f","Type":"ContainerDied","Data":"feb6caaa0f33a1208f010f5028985540de3a04d323e044a070682bbaee919f6d"} Jul 22 20:38:45.387277 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:45.387249 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" Jul 22 20:38:45.502081 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:45.502002 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7edb7b62-210d-463c-956c-028b89c3f28f-maas-api-service-ca\") pod \"7edb7b62-210d-463c-956c-028b89c3f28f\" (UID: \"7edb7b62-210d-463c-956c-028b89c3f28f\") " Jul 22 20:38:45.502353 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:45.502331 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7edb7b62-210d-463c-956c-028b89c3f28f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7edb7b62-210d-463c-956c-028b89c3f28f" (UID: "7edb7b62-210d-463c-956c-028b89c3f28f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:38:45.603483 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:45.603447 2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7edb7b62-210d-463c-956c-028b89c3f28f-maas-api-service-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:38:46.245825 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:46.245796 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" Jul 22 20:38:46.246010 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:46.245838 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l" event={"ID":"7edb7b62-210d-463c-956c-028b89c3f28f","Type":"ContainerDied","Data":"e0498be8f8ee414b052a66aeea782567e1bbfff7c73a5342f9da68fe0966502f"} Jul 22 20:38:46.246010 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:38:46.245894 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0498be8f8ee414b052a66aeea782567e1bbfff7c73a5342f9da68fe0966502f" Jul 22 20:39:02.346924 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:39:02.346837 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l"] Jul 22 20:39:02.349665 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:39:02.349639 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e950683d-4vp6l"] Jul 22 20:39:03.007700 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:39:03.007665 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7edb7b62-210d-463c-956c-028b89c3f28f" path="/var/lib/kubelet/pods/7edb7b62-210d-463c-956c-028b89c3f28f/volumes" Jul 22 20:41:31.181975 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:41:31.181856 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:41:31.198243 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:41:31.198220 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:42:03.584601 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.584519 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475"] Jul 22 20:42:03.585069 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.585016 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7edb7b62-210d-463c-956c-028b89c3f28f" containerName="revoke-keys" Jul 22 20:42:03.585069 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.585029 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="7edb7b62-210d-463c-956c-028b89c3f28f" containerName="revoke-keys" Jul 22 20:42:03.585147 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.585102 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="7edb7b62-210d-463c-956c-028b89c3f28f" containerName="revoke-keys" Jul 22 20:42:03.588324 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.588304 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" Jul 22 20:42:03.590618 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.590593 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hjqt7\"" Jul 22 20:42:03.595517 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.595484 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475"] Jul 22 20:42:03.716796 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.716755 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c07e65b-c963-470f-90d3-3ef716fc11c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475\" (UID: \"0c07e65b-c963-470f-90d3-3ef716fc11c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" Jul 22 20:42:03.817678 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.817640 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c07e65b-c963-470f-90d3-3ef716fc11c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475\" (UID: \"0c07e65b-c963-470f-90d3-3ef716fc11c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" Jul 22 20:42:03.818285 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.818265 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c07e65b-c963-470f-90d3-3ef716fc11c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475\" (UID: \"0c07e65b-c963-470f-90d3-3ef716fc11c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" Jul 22 20:42:03.901012 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:03.900912 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" Jul 22 20:42:04.047080 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:04.047053 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475"] Jul 22 20:42:04.049090 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:42:04.049059 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c07e65b_c963_470f_90d3_3ef716fc11c5.slice/crio-136d9709ac1c17f6591ea5f465380ccb3578a47267b25d28844439336486e419 WatchSource:0}: Error finding container 136d9709ac1c17f6591ea5f465380ccb3578a47267b25d28844439336486e419: Status 404 returned error can't find the container with id 136d9709ac1c17f6591ea5f465380ccb3578a47267b25d28844439336486e419 Jul 22 20:42:04.501094 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:04.501062 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" event={"ID":"0c07e65b-c963-470f-90d3-3ef716fc11c5","Type":"ContainerStarted","Data":"e7d2af09e597b0e5e546f2b9e8f3b2ef874eed843c0ee451aa50b1e0694c78b8"} Jul 22 20:42:04.501304 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:04.501118 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" event={"ID":"0c07e65b-c963-470f-90d3-3ef716fc11c5","Type":"ContainerStarted","Data":"136d9709ac1c17f6591ea5f465380ccb3578a47267b25d28844439336486e419"} Jul 22 20:42:04.515358 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:04.515302 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" podStartSLOduration=1.515285029 podStartE2EDuration="1.515285029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:42:04.513487207 +0000 UTC m=+2434.070230902" watchObservedRunningTime="2026-07-22 20:42:04.515285029 +0000 UTC m=+2434.072028726" Jul 22 20:42:05.508132 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:05.508100 2553 generic.go:356] "Generic (PLEG): container finished" podID="0c07e65b-c963-470f-90d3-3ef716fc11c5" containerID="e7d2af09e597b0e5e546f2b9e8f3b2ef874eed843c0ee451aa50b1e0694c78b8" exitCode=0 Jul 22 20:42:05.508545 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:05.508185 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" event={"ID":"0c07e65b-c963-470f-90d3-3ef716fc11c5","Type":"ContainerDied","Data":"e7d2af09e597b0e5e546f2b9e8f3b2ef874eed843c0ee451aa50b1e0694c78b8"} Jul 22 20:42:06.660797 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:06.660765 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" Jul 22 20:42:06.745218 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:06.745183 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c07e65b-c963-470f-90d3-3ef716fc11c5-maas-api-service-ca\") pod \"0c07e65b-c963-470f-90d3-3ef716fc11c5\" (UID: \"0c07e65b-c963-470f-90d3-3ef716fc11c5\") " Jul 22 20:42:06.745615 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:06.745580 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c07e65b-c963-470f-90d3-3ef716fc11c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c07e65b-c963-470f-90d3-3ef716fc11c5" (UID: "0c07e65b-c963-470f-90d3-3ef716fc11c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 20:42:06.846768 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:06.846679 2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c07e65b-c963-470f-90d3-3ef716fc11c5-maas-api-service-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:42:07.522563 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:07.522529 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" Jul 22 20:42:07.522563 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:07.522538 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475" event={"ID":"0c07e65b-c963-470f-90d3-3ef716fc11c5","Type":"ContainerDied","Data":"136d9709ac1c17f6591ea5f465380ccb3578a47267b25d28844439336486e419"} Jul 22 20:42:07.522774 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:07.522586 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="136d9709ac1c17f6591ea5f465380ccb3578a47267b25d28844439336486e419" Jul 22 20:42:18.658388 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:18.658350 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475"] Jul 22 20:42:18.661329 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:18.661300 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ba38bd-7t475"] Jul 22 20:42:19.007350 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:42:19.007317 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c07e65b-c963-470f-90d3-3ef716fc11c5" path="/var/lib/kubelet/pods/0c07e65b-c963-470f-90d3-3ef716fc11c5/volumes" Jul 22 20:45:31.292290 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:45:31.292206 2553 scope.go:117] "RemoveContainer" containerID="feb6caaa0f33a1208f010f5028985540de3a04d323e044a070682bbaee919f6d" Jul 22 20:46:31.232738 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:46:31.232709 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:46:31.253066 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:46:31.253044 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:48:31.333711 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:48:31.333678 2553 scope.go:117] "RemoveContainer" containerID="e7d2af09e597b0e5e546f2b9e8f3b2ef874eed843c0ee451aa50b1e0694c78b8" Jul 22 20:49:46.578829 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.578792 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7838-139711889376432"] Jul 22 20:49:46.579322 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.579306 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c07e65b-c963-470f-90d3-3ef716fc11c5" containerName="revoke-keys" Jul 22 20:49:46.579368 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.579324 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c07e65b-c963-470f-90d3-3ef716fc11c5" containerName="revoke-keys" Jul 22 20:49:46.579477 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.579454 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c07e65b-c963-470f-90d3-3ef716fc11c5" containerName="revoke-keys" Jul 22 20:49:46.582857 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.582839 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889376432" Jul 22 20:49:46.585100 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.585078 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nd6mq\"" Jul 22 20:49:46.588120 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.588097 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889376432"] Jul 22 20:49:46.684208 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.684174 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zlvh\" (UniqueName: \"kubernetes.io/projected/594a9f96-c226-43ff-a985-5c0e2c1a95cb-kube-api-access-7zlvh\") pod \"test-curl-7838-139711889376432\" (UID: \"594a9f96-c226-43ff-a985-5c0e2c1a95cb\") " pod="opendatahub/test-curl-7838-139711889376432" Jul 22 20:49:46.785395 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.785359 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zlvh\" (UniqueName: \"kubernetes.io/projected/594a9f96-c226-43ff-a985-5c0e2c1a95cb-kube-api-access-7zlvh\") pod \"test-curl-7838-139711889376432\" (UID: \"594a9f96-c226-43ff-a985-5c0e2c1a95cb\") " pod="opendatahub/test-curl-7838-139711889376432" Jul 22 20:49:46.794245 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.794218 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zlvh\" (UniqueName: \"kubernetes.io/projected/594a9f96-c226-43ff-a985-5c0e2c1a95cb-kube-api-access-7zlvh\") pod \"test-curl-7838-139711889376432\" (UID: \"594a9f96-c226-43ff-a985-5c0e2c1a95cb\") " pod="opendatahub/test-curl-7838-139711889376432" Jul 22 20:49:46.893765 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:46.893690 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889376432" Jul 22 20:49:47.036483 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:47.036458 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889376432"] Jul 22 20:49:47.038311 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:49:47.038286 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod594a9f96_c226_43ff_a985_5c0e2c1a95cb.slice/crio-444943ff3f1adc8ac2c90460ff943f88d821b775603ede5c46a90c74b12028d7 WatchSource:0}: Error finding container 444943ff3f1adc8ac2c90460ff943f88d821b775603ede5c46a90c74b12028d7: Status 404 returned error can't find the container with id 444943ff3f1adc8ac2c90460ff943f88d821b775603ede5c46a90c74b12028d7 Jul 22 20:49:47.039732 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:47.039711 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 20:49:47.455580 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:47.455537 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889376432" event={"ID":"594a9f96-c226-43ff-a985-5c0e2c1a95cb","Type":"ContainerStarted","Data":"444943ff3f1adc8ac2c90460ff943f88d821b775603ede5c46a90c74b12028d7"} Jul 22 20:49:48.464060 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:48.464023 2553 generic.go:356] "Generic (PLEG): container finished" podID="594a9f96-c226-43ff-a985-5c0e2c1a95cb" containerID="a693088c94dfb91d7595da1698af34e1dae0fa2ed61ced94dd6e08f3dbb417d1" exitCode=0 Jul 22 20:49:48.464060 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:48.464072 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889376432" event={"ID":"594a9f96-c226-43ff-a985-5c0e2c1a95cb","Type":"ContainerDied","Data":"a693088c94dfb91d7595da1698af34e1dae0fa2ed61ced94dd6e08f3dbb417d1"} Jul 22 20:49:49.613675 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.613650 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889376432" Jul 22 20:49:49.647668 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.647632 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7838-139711889376432_594a9f96-c226-43ff-a985-5c0e2c1a95cb/test-curl-7838-139711889376432/0.log" Jul 22 20:49:49.672595 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.672564 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7838-139711889376432"] Jul 22 20:49:49.676634 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.676611 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7838-139711889376432"] Jul 22 20:49:49.713895 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.713871 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zlvh\" (UniqueName: \"kubernetes.io/projected/594a9f96-c226-43ff-a985-5c0e2c1a95cb-kube-api-access-7zlvh\") pod \"594a9f96-c226-43ff-a985-5c0e2c1a95cb\" (UID: \"594a9f96-c226-43ff-a985-5c0e2c1a95cb\") " Jul 22 20:49:49.715931 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.715898 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/594a9f96-c226-43ff-a985-5c0e2c1a95cb-kube-api-access-7zlvh" (OuterVolumeSpecName: "kube-api-access-7zlvh") pod "594a9f96-c226-43ff-a985-5c0e2c1a95cb" (UID: "594a9f96-c226-43ff-a985-5c0e2c1a95cb"). InnerVolumeSpecName "kube-api-access-7zlvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:49:49.806034 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.805993 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7838-139711889378224"] Jul 22 20:49:49.806559 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.806540 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="594a9f96-c226-43ff-a985-5c0e2c1a95cb" containerName="test-curl-7838-139711889376432" Jul 22 20:49:49.806648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.806560 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="594a9f96-c226-43ff-a985-5c0e2c1a95cb" containerName="test-curl-7838-139711889376432" Jul 22 20:49:49.806728 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.806712 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="594a9f96-c226-43ff-a985-5c0e2c1a95cb" containerName="test-curl-7838-139711889376432" Jul 22 20:49:49.809907 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.809887 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889378224" Jul 22 20:49:49.814767 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.814745 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zlvh\" (UniqueName: \"kubernetes.io/projected/594a9f96-c226-43ff-a985-5c0e2c1a95cb-kube-api-access-7zlvh\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:49:49.818276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.818254 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889378224"] Jul 22 20:49:49.916200 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:49.916157 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86bch\" (UniqueName: \"kubernetes.io/projected/9e97e982-90a2-4fdf-9de3-faf7fd5beabb-kube-api-access-86bch\") pod \"test-curl-7838-139711889378224\" (UID: \"9e97e982-90a2-4fdf-9de3-faf7fd5beabb\") " pod="opendatahub/test-curl-7838-139711889378224" Jul 22 20:49:50.017707 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:50.017629 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86bch\" (UniqueName: \"kubernetes.io/projected/9e97e982-90a2-4fdf-9de3-faf7fd5beabb-kube-api-access-86bch\") pod \"test-curl-7838-139711889378224\" (UID: \"9e97e982-90a2-4fdf-9de3-faf7fd5beabb\") " pod="opendatahub/test-curl-7838-139711889378224" Jul 22 20:49:50.027828 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:50.027799 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86bch\" (UniqueName: \"kubernetes.io/projected/9e97e982-90a2-4fdf-9de3-faf7fd5beabb-kube-api-access-86bch\") pod \"test-curl-7838-139711889378224\" (UID: \"9e97e982-90a2-4fdf-9de3-faf7fd5beabb\") " pod="opendatahub/test-curl-7838-139711889378224" Jul 22 20:49:50.120070 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:50.120035 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889378224" Jul 22 20:49:50.257271 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:50.257246 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889378224"] Jul 22 20:49:50.259430 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:49:50.259383 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e97e982_90a2_4fdf_9de3_faf7fd5beabb.slice/crio-5258354eb2eb16cd909ff8bffb04f4363a0ecba00de4873320de674bc0d06a9b WatchSource:0}: Error finding container 5258354eb2eb16cd909ff8bffb04f4363a0ecba00de4873320de674bc0d06a9b: Status 404 returned error can't find the container with id 5258354eb2eb16cd909ff8bffb04f4363a0ecba00de4873320de674bc0d06a9b Jul 22 20:49:50.478554 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:50.478516 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889378224" event={"ID":"9e97e982-90a2-4fdf-9de3-faf7fd5beabb","Type":"ContainerStarted","Data":"5258354eb2eb16cd909ff8bffb04f4363a0ecba00de4873320de674bc0d06a9b"} Jul 22 20:49:50.479816 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:50.479791 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889376432" Jul 22 20:49:50.479816 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:50.479805 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="444943ff3f1adc8ac2c90460ff943f88d821b775603ede5c46a90c74b12028d7" Jul 22 20:49:50.490508 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:49:50.490478 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7838-139711889376432\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="594a9f96-c226-43ff-a985-5c0e2c1a95cb" pod="opendatahub/test-curl-7838-139711889376432" Jul 22 20:49:51.006706 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:49:51.006671 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7838-139711889376432\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="594a9f96-c226-43ff-a985-5c0e2c1a95cb" pod="opendatahub/test-curl-7838-139711889376432" Jul 22 20:49:51.009307 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:51.009281 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="594a9f96-c226-43ff-a985-5c0e2c1a95cb" path="/var/lib/kubelet/pods/594a9f96-c226-43ff-a985-5c0e2c1a95cb/volumes" Jul 22 20:49:51.488368 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:51.488332 2553 generic.go:356] "Generic (PLEG): container finished" podID="9e97e982-90a2-4fdf-9de3-faf7fd5beabb" containerID="7c35419f0cb5124faa258a4e48b00bf119a41452888b5fd073aae9fa102154b9" exitCode=0 Jul 22 20:49:51.488544 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:51.488433 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889378224" event={"ID":"9e97e982-90a2-4fdf-9de3-faf7fd5beabb","Type":"ContainerDied","Data":"7c35419f0cb5124faa258a4e48b00bf119a41452888b5fd073aae9fa102154b9"} Jul 22 20:49:52.639176 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.639147 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889378224" Jul 22 20:49:52.657988 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.657960 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7838-139711889378224_9e97e982-90a2-4fdf-9de3-faf7fd5beabb/test-curl-7838-139711889378224/0.log" Jul 22 20:49:52.681094 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.681067 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7838-139711889378224"] Jul 22 20:49:52.684019 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.683997 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7838-139711889378224"] Jul 22 20:49:52.742766 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.742728 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86bch\" (UniqueName: \"kubernetes.io/projected/9e97e982-90a2-4fdf-9de3-faf7fd5beabb-kube-api-access-86bch\") pod \"9e97e982-90a2-4fdf-9de3-faf7fd5beabb\" (UID: \"9e97e982-90a2-4fdf-9de3-faf7fd5beabb\") " Jul 22 20:49:52.744821 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.744793 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e97e982-90a2-4fdf-9de3-faf7fd5beabb-kube-api-access-86bch" (OuterVolumeSpecName: "kube-api-access-86bch") pod "9e97e982-90a2-4fdf-9de3-faf7fd5beabb" (UID: "9e97e982-90a2-4fdf-9de3-faf7fd5beabb"). InnerVolumeSpecName "kube-api-access-86bch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:49:52.843971 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.843937 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86bch\" (UniqueName: \"kubernetes.io/projected/9e97e982-90a2-4fdf-9de3-faf7fd5beabb-kube-api-access-86bch\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:49:52.869750 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.869717 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:52.870190 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.870177 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e97e982-90a2-4fdf-9de3-faf7fd5beabb" containerName="test-curl-7838-139711889378224" Jul 22 20:49:52.870231 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.870191 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e97e982-90a2-4fdf-9de3-faf7fd5beabb" containerName="test-curl-7838-139711889378224" Jul 22 20:49:52.870278 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.870269 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e97e982-90a2-4fdf-9de3-faf7fd5beabb" containerName="test-curl-7838-139711889378224" Jul 22 20:49:52.873443 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.873399 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:52.878795 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.878771 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:52.944814 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:52.944778 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpbj4\" (UniqueName: \"kubernetes.io/projected/fbeb1d07-b9ae-45ef-8988-3b67bf7ab142-kube-api-access-cpbj4\") pod \"test-curl-7838-139711889466256\" (UID: \"fbeb1d07-b9ae-45ef-8988-3b67bf7ab142\") " pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:53.007792 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:53.007717 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e97e982-90a2-4fdf-9de3-faf7fd5beabb" path="/var/lib/kubelet/pods/9e97e982-90a2-4fdf-9de3-faf7fd5beabb/volumes" Jul 22 20:49:53.046066 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:53.046037 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpbj4\" (UniqueName: \"kubernetes.io/projected/fbeb1d07-b9ae-45ef-8988-3b67bf7ab142-kube-api-access-cpbj4\") pod \"test-curl-7838-139711889466256\" (UID: \"fbeb1d07-b9ae-45ef-8988-3b67bf7ab142\") " pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:53.054255 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:53.054230 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpbj4\" (UniqueName: \"kubernetes.io/projected/fbeb1d07-b9ae-45ef-8988-3b67bf7ab142-kube-api-access-cpbj4\") pod \"test-curl-7838-139711889466256\" (UID: \"fbeb1d07-b9ae-45ef-8988-3b67bf7ab142\") " pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:53.184642 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:53.184606 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:53.323874 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:53.323841 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:53.324588 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:49:53.324563 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbeb1d07_b9ae_45ef_8988_3b67bf7ab142.slice/crio-01e0a861d7cc479cded62e029a372e3fb227984ed1bfe44b483e006d6f576ff7 WatchSource:0}: Error finding container 01e0a861d7cc479cded62e029a372e3fb227984ed1bfe44b483e006d6f576ff7: Status 404 returned error can't find the container with id 01e0a861d7cc479cded62e029a372e3fb227984ed1bfe44b483e006d6f576ff7 Jul 22 20:49:53.502648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:53.502614 2553 scope.go:117] "RemoveContainer" containerID="7c35419f0cb5124faa258a4e48b00bf119a41452888b5fd073aae9fa102154b9" Jul 22 20:49:53.502846 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:53.502626 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889378224" Jul 22 20:49:53.504045 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:53.504021 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889466256" event={"ID":"fbeb1d07-b9ae-45ef-8988-3b67bf7ab142","Type":"ContainerStarted","Data":"01e0a861d7cc479cded62e029a372e3fb227984ed1bfe44b483e006d6f576ff7"} Jul 22 20:49:54.512757 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:54.512718 2553 generic.go:356] "Generic (PLEG): container finished" podID="fbeb1d07-b9ae-45ef-8988-3b67bf7ab142" containerID="1596027fc5d59026ff0c809edd3568e580fd72397f70d15a72f409cc66923e38" exitCode=0 Jul 22 20:49:54.513184 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:54.512818 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889466256" event={"ID":"fbeb1d07-b9ae-45ef-8988-3b67bf7ab142","Type":"ContainerDied","Data":"1596027fc5d59026ff0c809edd3568e580fd72397f70d15a72f409cc66923e38"} Jul 22 20:49:55.658665 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.658641 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:55.682570 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.682544 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7838-139711889466256_fbeb1d07-b9ae-45ef-8988-3b67bf7ab142/test-curl-7838-139711889466256/0.log" Jul 22 20:49:55.706903 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.706864 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:55.713262 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.713234 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:55.775200 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.775114 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpbj4\" (UniqueName: \"kubernetes.io/projected/fbeb1d07-b9ae-45ef-8988-3b67bf7ab142-kube-api-access-cpbj4\") pod \"fbeb1d07-b9ae-45ef-8988-3b67bf7ab142\" (UID: \"fbeb1d07-b9ae-45ef-8988-3b67bf7ab142\") " Jul 22 20:49:55.777174 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.777145 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbeb1d07-b9ae-45ef-8988-3b67bf7ab142-kube-api-access-cpbj4" (OuterVolumeSpecName: "kube-api-access-cpbj4") pod "fbeb1d07-b9ae-45ef-8988-3b67bf7ab142" (UID: "fbeb1d07-b9ae-45ef-8988-3b67bf7ab142"). InnerVolumeSpecName "kube-api-access-cpbj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:49:55.836591 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.836562 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:55.837077 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.837062 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbeb1d07-b9ae-45ef-8988-3b67bf7ab142" containerName="test-curl-7838-139711889466256" Jul 22 20:49:55.837114 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.837078 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbeb1d07-b9ae-45ef-8988-3b67bf7ab142" containerName="test-curl-7838-139711889466256" Jul 22 20:49:55.837162 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.837153 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbeb1d07-b9ae-45ef-8988-3b67bf7ab142" containerName="test-curl-7838-139711889466256" Jul 22 20:49:55.840461 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.840442 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:55.846290 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.846268 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:55.876480 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.876455 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9blm\" (UniqueName: \"kubernetes.io/projected/8548513f-ee79-476b-b917-80358e5418c6-kube-api-access-x9blm\") pod \"test-curl-7838-139711889466256\" (UID: \"8548513f-ee79-476b-b917-80358e5418c6\") " pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:55.876583 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.876560 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpbj4\" (UniqueName: \"kubernetes.io/projected/fbeb1d07-b9ae-45ef-8988-3b67bf7ab142-kube-api-access-cpbj4\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:49:55.978039 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.978000 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9blm\" (UniqueName: \"kubernetes.io/projected/8548513f-ee79-476b-b917-80358e5418c6-kube-api-access-x9blm\") pod \"test-curl-7838-139711889466256\" (UID: \"8548513f-ee79-476b-b917-80358e5418c6\") " pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:55.987272 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:55.987240 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9blm\" (UniqueName: \"kubernetes.io/projected/8548513f-ee79-476b-b917-80358e5418c6-kube-api-access-x9blm\") pod \"test-curl-7838-139711889466256\" (UID: \"8548513f-ee79-476b-b917-80358e5418c6\") " pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:56.151959 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:56.151880 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:56.286924 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:56.286894 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:56.288063 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:49:56.288036 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8548513f_ee79_476b_b917_80358e5418c6.slice/crio-ff35bac0503937d8897dda7e3b39bb12fc618f6fc5db32fe92873f682ec40d39 WatchSource:0}: Error finding container ff35bac0503937d8897dda7e3b39bb12fc618f6fc5db32fe92873f682ec40d39: Status 404 returned error can't find the container with id ff35bac0503937d8897dda7e3b39bb12fc618f6fc5db32fe92873f682ec40d39 Jul 22 20:49:56.527058 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:56.527021 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889466256" event={"ID":"8548513f-ee79-476b-b917-80358e5418c6","Type":"ContainerStarted","Data":"ff35bac0503937d8897dda7e3b39bb12fc618f6fc5db32fe92873f682ec40d39"} Jul 22 20:49:56.528234 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:56.528211 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01e0a861d7cc479cded62e029a372e3fb227984ed1bfe44b483e006d6f576ff7" Jul 22 20:49:56.528306 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:56.528236 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:56.540501 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:56.540467 2553 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7838-139711889466256" oldPodUID="fbeb1d07-b9ae-45ef-8988-3b67bf7ab142" podUID="8548513f-ee79-476b-b917-80358e5418c6" Jul 22 20:49:57.007371 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:57.007338 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbeb1d07-b9ae-45ef-8988-3b67bf7ab142" path="/var/lib/kubelet/pods/fbeb1d07-b9ae-45ef-8988-3b67bf7ab142/volumes" Jul 22 20:49:57.535374 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:57.535343 2553 generic.go:356] "Generic (PLEG): container finished" podID="8548513f-ee79-476b-b917-80358e5418c6" containerID="15cbd0aa15746a743ca10a0611ea78de5639bada2a2e07e709604369575986f2" exitCode=0 Jul 22 20:49:57.535600 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:57.535438 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889466256" event={"ID":"8548513f-ee79-476b-b917-80358e5418c6","Type":"ContainerDied","Data":"15cbd0aa15746a743ca10a0611ea78de5639bada2a2e07e709604369575986f2"} Jul 22 20:49:58.680980 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.680956 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:58.703169 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.703145 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7838-139711889466256_8548513f-ee79-476b-b917-80358e5418c6/test-curl-7838-139711889466256/0.log" Jul 22 20:49:58.729536 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.729499 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:58.732149 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.732127 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7838-139711889466256"] Jul 22 20:49:58.805575 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.805473 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9blm\" (UniqueName: \"kubernetes.io/projected/8548513f-ee79-476b-b917-80358e5418c6-kube-api-access-x9blm\") pod \"8548513f-ee79-476b-b917-80358e5418c6\" (UID: \"8548513f-ee79-476b-b917-80358e5418c6\") " Jul 22 20:49:58.807625 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.807595 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8548513f-ee79-476b-b917-80358e5418c6-kube-api-access-x9blm" (OuterVolumeSpecName: "kube-api-access-x9blm") pod "8548513f-ee79-476b-b917-80358e5418c6" (UID: "8548513f-ee79-476b-b917-80358e5418c6"). InnerVolumeSpecName "kube-api-access-x9blm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:49:58.855902 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.855867 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7838-139711889467264"] Jul 22 20:49:58.856348 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.856334 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8548513f-ee79-476b-b917-80358e5418c6" containerName="test-curl-7838-139711889466256" Jul 22 20:49:58.856391 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.856350 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8548513f-ee79-476b-b917-80358e5418c6" containerName="test-curl-7838-139711889466256" Jul 22 20:49:58.856452 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.856439 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="8548513f-ee79-476b-b917-80358e5418c6" containerName="test-curl-7838-139711889466256" Jul 22 20:49:58.859545 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.859529 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889467264" Jul 22 20:49:58.878059 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.878019 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889467264"] Jul 22 20:49:58.906334 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.906295 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gdfv\" (UniqueName: \"kubernetes.io/projected/36be5d93-0d8f-49c6-a2f2-e337ce239736-kube-api-access-7gdfv\") pod \"test-curl-7838-139711889467264\" (UID: \"36be5d93-0d8f-49c6-a2f2-e337ce239736\") " pod="opendatahub/test-curl-7838-139711889467264" Jul 22 20:49:58.906522 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:58.906495 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9blm\" (UniqueName: \"kubernetes.io/projected/8548513f-ee79-476b-b917-80358e5418c6-kube-api-access-x9blm\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:49:59.007336 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:59.007302 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gdfv\" (UniqueName: \"kubernetes.io/projected/36be5d93-0d8f-49c6-a2f2-e337ce239736-kube-api-access-7gdfv\") pod \"test-curl-7838-139711889467264\" (UID: \"36be5d93-0d8f-49c6-a2f2-e337ce239736\") " pod="opendatahub/test-curl-7838-139711889467264" Jul 22 20:49:59.008585 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:59.008562 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8548513f-ee79-476b-b917-80358e5418c6" path="/var/lib/kubelet/pods/8548513f-ee79-476b-b917-80358e5418c6/volumes" Jul 22 20:49:59.017093 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:59.017068 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gdfv\" (UniqueName: \"kubernetes.io/projected/36be5d93-0d8f-49c6-a2f2-e337ce239736-kube-api-access-7gdfv\") pod \"test-curl-7838-139711889467264\" (UID: \"36be5d93-0d8f-49c6-a2f2-e337ce239736\") " pod="opendatahub/test-curl-7838-139711889467264" Jul 22 20:49:59.182188 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:59.182098 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889467264" Jul 22 20:49:59.323169 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:49:59.323134 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36be5d93_0d8f_49c6_a2f2_e337ce239736.slice/crio-69b64781c41aca1d116d447c4b6ba3cb36251d27bcf6afa7807f2be5aecb36dc WatchSource:0}: Error finding container 69b64781c41aca1d116d447c4b6ba3cb36251d27bcf6afa7807f2be5aecb36dc: Status 404 returned error can't find the container with id 69b64781c41aca1d116d447c4b6ba3cb36251d27bcf6afa7807f2be5aecb36dc Jul 22 20:49:59.323169 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:59.323161 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7838-139711889467264"] Jul 22 20:49:59.548873 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:59.548838 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889466256" Jul 22 20:49:59.549076 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:59.548845 2553 scope.go:117] "RemoveContainer" containerID="15cbd0aa15746a743ca10a0611ea78de5639bada2a2e07e709604369575986f2" Jul 22 20:49:59.550531 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:49:59.550202 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889467264" event={"ID":"36be5d93-0d8f-49c6-a2f2-e337ce239736","Type":"ContainerStarted","Data":"69b64781c41aca1d116d447c4b6ba3cb36251d27bcf6afa7807f2be5aecb36dc"} Jul 22 20:50:00.559603 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:00.559566 2553 generic.go:356] "Generic (PLEG): container finished" podID="36be5d93-0d8f-49c6-a2f2-e337ce239736" containerID="97ab20efc5a202954f877f1b5aa70b1804f3f3e1135397ec575a32d3e05c2aef" exitCode=0 Jul 22 20:50:00.559975 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:00.559646 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7838-139711889467264" event={"ID":"36be5d93-0d8f-49c6-a2f2-e337ce239736","Type":"ContainerDied","Data":"97ab20efc5a202954f877f1b5aa70b1804f3f3e1135397ec575a32d3e05c2aef"} Jul 22 20:50:01.707867 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:01.707844 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889467264" Jul 22 20:50:01.732239 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:01.732213 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7838-139711889467264_36be5d93-0d8f-49c6-a2f2-e337ce239736/test-curl-7838-139711889467264/0.log" Jul 22 20:50:01.763867 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:01.763838 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7838-139711889467264"] Jul 22 20:50:01.768296 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:01.768270 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7838-139711889467264"] Jul 22 20:50:01.831430 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:01.831329 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gdfv\" (UniqueName: \"kubernetes.io/projected/36be5d93-0d8f-49c6-a2f2-e337ce239736-kube-api-access-7gdfv\") pod \"36be5d93-0d8f-49c6-a2f2-e337ce239736\" (UID: \"36be5d93-0d8f-49c6-a2f2-e337ce239736\") " Jul 22 20:50:01.833366 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:01.833335 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36be5d93-0d8f-49c6-a2f2-e337ce239736-kube-api-access-7gdfv" (OuterVolumeSpecName: "kube-api-access-7gdfv") pod "36be5d93-0d8f-49c6-a2f2-e337ce239736" (UID: "36be5d93-0d8f-49c6-a2f2-e337ce239736"). InnerVolumeSpecName "kube-api-access-7gdfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 20:50:01.932645 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:01.932610 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gdfv\" (UniqueName: \"kubernetes.io/projected/36be5d93-0d8f-49c6-a2f2-e337ce239736-kube-api-access-7gdfv\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 22 20:50:02.573886 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:02.573854 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7838-139711889467264" Jul 22 20:50:02.573886 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:02.573877 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69b64781c41aca1d116d447c4b6ba3cb36251d27bcf6afa7807f2be5aecb36dc" Jul 22 20:50:02.584074 ip-10-0-139-68 kubenswrapper[2553]: E0722 20:50:02.584043 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7838-139711889467264\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="36be5d93-0d8f-49c6-a2f2-e337ce239736" pod="opendatahub/test-curl-7838-139711889467264" Jul 22 20:50:03.007729 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:03.007694 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36be5d93-0d8f-49c6-a2f2-e337ce239736" path="/var/lib/kubelet/pods/36be5d93-0d8f-49c6-a2f2-e337ce239736/volumes" Jul 22 20:50:55.557651 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:55.557559 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vn5kw_e5b35131-90f5-4246-9ba5-9c72cb136c00/manager/0.log" Jul 22 20:50:55.687391 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:55.687328 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2wrrs_dead0468-003c-4f71-8848-5b79e506e482/manager/0.log" Jul 22 20:50:55.935982 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:55.935907 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dw6vc_d11451bb-b7fa-4604-887e-8c16f46c73c1/server/0.log" Jul 22 20:50:56.061206 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:56.061168 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-857jb_85233d48-b20f-4981-a4ca-19d228a4e537/manager/0.log" Jul 22 20:50:56.480183 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:56.480154 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc56f4f-zwzkl_f0b1a600-adef-40bf-b36f-e99c5f0d9db5/copy-manifests/0.log" Jul 22 20:50:56.486848 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:56.486812 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc56f4f-zwzkl_f0b1a600-adef-40bf-b36f-e99c5f0d9db5/manager/0.log" Jul 22 20:50:56.846078 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:56.845988 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5d57d6b457-6fj26_0b17580e-b0cd-45a0-b536-07bcf3563ace/maas-api/0.log" Jul 22 20:50:56.951132 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:56.951101 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745855-72nrg_3e7509fa-bf34-492d-b9e7-09a8cb3ddaf1/cleanup/0.log" Jul 22 20:50:57.293573 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:57.293540 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-z74nc_80318c2e-2e37-4430-8e44-1715aea5e47a/postgres/0.log" Jul 22 20:50:58.087561 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.087531 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25_4f6d69b8-8d18-4ad1-9fa0-c82bf8604500/util/0.log" Jul 22 20:50:58.093573 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.093545 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25_4f6d69b8-8d18-4ad1-9fa0-c82bf8604500/pull/0.log" Jul 22 20:50:58.098881 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.098856 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25_4f6d69b8-8d18-4ad1-9fa0-c82bf8604500/extract/0.log" Jul 22 20:50:58.218830 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.218797 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs_fdceed12-3f3c-4c8b-ad29-03c60e25e034/pull/0.log" Jul 22 20:50:58.225962 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.225930 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs_fdceed12-3f3c-4c8b-ad29-03c60e25e034/extract/0.log" Jul 22 20:50:58.232993 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.232970 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs_fdceed12-3f3c-4c8b-ad29-03c60e25e034/util/0.log" Jul 22 20:50:58.349363 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.349283 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz_aa322f41-c382-4b42-9a9a-e2e9258aa5b0/util/0.log" Jul 22 20:50:58.355841 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.355812 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz_aa322f41-c382-4b42-9a9a-e2e9258aa5b0/pull/0.log" Jul 22 20:50:58.362280 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.362258 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz_aa322f41-c382-4b42-9a9a-e2e9258aa5b0/extract/0.log" Jul 22 20:50:58.477815 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.477788 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n_2094f4b5-883b-4788-ab66-e971a541002a/util/0.log" Jul 22 20:50:58.487597 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.487571 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n_2094f4b5-883b-4788-ab66-e971a541002a/pull/0.log" Jul 22 20:50:58.494001 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.493981 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n_2094f4b5-883b-4788-ab66-e971a541002a/extract/0.log" Jul 22 20:50:58.741936 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.741909 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2ftmg_3348f631-5957-4bfe-ac51-428961c97785/manager/0.log" Jul 22 20:50:58.883211 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.883177 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bdqms_ae794a3f-1e94-4d46-a267-20384eb124a5/manager/0.log" Jul 22 20:50:58.994606 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:58.994516 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4ltjd_2e14b33a-43c9-4a0d-8487-661c149da3c1/kuadrant-console-plugin/0.log" Jul 22 20:50:59.119600 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:59.119573 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-v8xph_8e2dac47-0347-4473-9744-7eeefb4ac48a/registry-server/0.log" Jul 22 20:50:59.487083 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:50:59.487057 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nlwsc_6fb5da73-00d8-4d50-a0b2-e0c6806102e9/manager/0.log" Jul 22 20:51:00.180888 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:00.180860 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-787b995ffc-vfsp7_ad8de055-bd52-4ba9-8b83-ac0845649cee/kube-auth-proxy/0.log" Jul 22 20:51:00.305001 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:00.304964 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qjgnt_ae4f0dfa-c828-4b82-bd55-ea4ef01369b8/istio-proxy/1.log" Jul 22 20:51:00.427427 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:00.427380 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-5qbzv_072bf681-5686-470f-8e3f-f13659a49b9d/payload-pre-processing/0.log" Jul 22 20:51:00.551307 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:00.551274 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-g47jq_2eb8d9e2-7b38-4512-8920-e1be5129916d/payload-processing/0.log" Jul 22 20:51:01.008377 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:01.008346 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-s4lcc_d849b31d-21d4-4b7e-859d-8ce6785a6e93/main/0.log" Jul 22 20:51:01.129683 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:01.129651 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-7cjd5_c79304af-762e-48be-bc44-c1b46659e551/main/0.log" Jul 22 20:51:01.239800 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:01.239770 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-clqhx_77a9a876-bdf9-495d-9e31-839c948ab7a8/main/0.log" Jul 22 20:51:01.482420 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:01.482370 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-qdvf5_91dabbf3-80a8-4f6c-a2a8-e5c43f07d4f3/main/0.log" Jul 22 20:51:10.951363 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:10.951329 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mxfbh_74d38ca2-5c25-4236-a7b5-335fc07b9ca2/global-pull-secret-syncer/0.log" Jul 22 20:51:11.093166 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:11.093130 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cmnc4_d5f9c0b2-40ce-4b3c-92df-28329cda5d03/konnectivity-agent/0.log" Jul 22 20:51:11.183753 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:11.183710 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-68.ec2.internal_1aa63fbc826c704af774266db16e0b0d/haproxy/0.log" Jul 22 20:51:15.943783 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:15.943728 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25_4f6d69b8-8d18-4ad1-9fa0-c82bf8604500/extract/0.log" Jul 22 20:51:15.966064 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:15.966038 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25_4f6d69b8-8d18-4ad1-9fa0-c82bf8604500/util/0.log" Jul 22 20:51:15.989513 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:15.989481 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599tm25_4f6d69b8-8d18-4ad1-9fa0-c82bf8604500/pull/0.log" Jul 22 20:51:16.025078 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.025046 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs_fdceed12-3f3c-4c8b-ad29-03c60e25e034/extract/0.log" Jul 22 20:51:16.046948 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.046921 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs_fdceed12-3f3c-4c8b-ad29-03c60e25e034/util/0.log" Jul 22 20:51:16.071668 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.071637 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05t8cs_fdceed12-3f3c-4c8b-ad29-03c60e25e034/pull/0.log" Jul 22 20:51:16.099146 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.099120 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz_aa322f41-c382-4b42-9a9a-e2e9258aa5b0/extract/0.log" Jul 22 20:51:16.121875 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.121845 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz_aa322f41-c382-4b42-9a9a-e2e9258aa5b0/util/0.log" Jul 22 20:51:16.144034 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.144004 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv5kz_aa322f41-c382-4b42-9a9a-e2e9258aa5b0/pull/0.log" Jul 22 20:51:16.172687 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.172652 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n_2094f4b5-883b-4788-ab66-e971a541002a/extract/0.log" Jul 22 20:51:16.193309 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.193279 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n_2094f4b5-883b-4788-ab66-e971a541002a/util/0.log" Jul 22 20:51:16.220440 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.220394 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kzl4n_2094f4b5-883b-4788-ab66-e971a541002a/pull/0.log" Jul 22 20:51:16.902690 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.902654 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2ftmg_3348f631-5957-4bfe-ac51-428961c97785/manager/0.log" Jul 22 20:51:16.928276 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.928237 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bdqms_ae794a3f-1e94-4d46-a267-20384eb124a5/manager/0.log" Jul 22 20:51:16.953684 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:16.953656 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4ltjd_2e14b33a-43c9-4a0d-8487-661c149da3c1/kuadrant-console-plugin/0.log" Jul 22 20:51:17.008126 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:17.008096 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-v8xph_8e2dac47-0347-4473-9744-7eeefb4ac48a/registry-server/0.log" Jul 22 20:51:17.363861 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:17.363828 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nlwsc_6fb5da73-00d8-4d50-a0b2-e0c6806102e9/manager/0.log" Jul 22 20:51:19.040201 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.040160 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249/alertmanager/0.log" Jul 22 20:51:19.061764 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.061735 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249/config-reloader/0.log" Jul 22 20:51:19.082794 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.082764 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249/kube-rbac-proxy-web/0.log" Jul 22 20:51:19.106136 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.106106 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249/kube-rbac-proxy/0.log" Jul 22 20:51:19.129340 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.129309 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249/kube-rbac-proxy-metric/0.log" Jul 22 20:51:19.151971 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.151945 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249/prom-label-proxy/0.log" Jul 22 20:51:19.178869 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.178847 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0acbcbe8-aeea-4ef9-b1e5-3ca9cef33249/init-config-reloader/0.log" Jul 22 20:51:19.254545 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.254514 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-6jqxp_995e6011-ac8e-49f7-91f8-b6c0841f130a/kube-state-metrics/0.log" Jul 22 20:51:19.270838 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.270809 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-6jqxp_995e6011-ac8e-49f7-91f8-b6c0841f130a/kube-rbac-proxy-main/0.log" Jul 22 20:51:19.292170 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.292077 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-6jqxp_995e6011-ac8e-49f7-91f8-b6c0841f130a/kube-rbac-proxy-self/0.log" Jul 22 20:51:19.522880 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.522850 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qlg52_11ce996a-40c1-4ee5-a4ea-19f07f6b89d8/node-exporter/0.log" Jul 22 20:51:19.541904 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.541874 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qlg52_11ce996a-40c1-4ee5-a4ea-19f07f6b89d8/kube-rbac-proxy/0.log" Jul 22 20:51:19.563733 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.563658 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qlg52_11ce996a-40c1-4ee5-a4ea-19f07f6b89d8/init-textfile/0.log" Jul 22 20:51:19.861579 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.861475 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-p9xw2_7080bba7-a4ad-4f85-bdd7-04c7b8539dc1/prometheus-operator-admission-webhook/0.log" Jul 22 20:51:19.961151 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.961092 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-779d5b5c4b-qkxf7_e474344e-cbbe-4ffc-86c3-9783990f3e83/thanos-query/0.log" Jul 22 20:51:19.980260 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:19.980230 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-779d5b5c4b-qkxf7_e474344e-cbbe-4ffc-86c3-9783990f3e83/kube-rbac-proxy-web/0.log" Jul 22 20:51:20.000723 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.000689 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-779d5b5c4b-qkxf7_e474344e-cbbe-4ffc-86c3-9783990f3e83/kube-rbac-proxy/0.log" Jul 22 20:51:20.022436 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.022392 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-779d5b5c4b-qkxf7_e474344e-cbbe-4ffc-86c3-9783990f3e83/prom-label-proxy/0.log" Jul 22 20:51:20.043559 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.043530 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-779d5b5c4b-qkxf7_e474344e-cbbe-4ffc-86c3-9783990f3e83/kube-rbac-proxy-rules/0.log" Jul 22 20:51:20.063611 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.063584 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-779d5b5c4b-qkxf7_e474344e-cbbe-4ffc-86c3-9783990f3e83/kube-rbac-proxy-metrics/0.log" Jul 22 20:51:20.696648 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.696609 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8"] Jul 22 20:51:20.697208 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.697187 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36be5d93-0d8f-49c6-a2f2-e337ce239736" containerName="test-curl-7838-139711889467264" Jul 22 20:51:20.697320 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.697211 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="36be5d93-0d8f-49c6-a2f2-e337ce239736" containerName="test-curl-7838-139711889467264" Jul 22 20:51:20.697378 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.697325 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="36be5d93-0d8f-49c6-a2f2-e337ce239736" containerName="test-curl-7838-139711889467264" Jul 22 20:51:20.700897 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.700875 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.703224 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.703196 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nzwv7\"/\"default-dockercfg-mc8gv\"" Jul 22 20:51:20.703354 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.703290 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nzwv7\"/\"openshift-service-ca.crt\"" Jul 22 20:51:20.703966 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.703947 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nzwv7\"/\"kube-root-ca.crt\"" Jul 22 20:51:20.712060 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.712030 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8"] Jul 22 20:51:20.790852 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.790810 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-sys\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.791041 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.790881 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-podres\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.791041 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.790933 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-proc\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.791041 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.790981 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b4jb\" (UniqueName: \"kubernetes.io/projected/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-kube-api-access-6b4jb\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.791041 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.791007 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-lib-modules\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.891857 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.891816 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-sys\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.892053 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.891869 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-podres\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.892053 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.891890 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-proc\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.892053 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.891915 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b4jb\" (UniqueName: \"kubernetes.io/projected/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-kube-api-access-6b4jb\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.892053 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.891936 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-lib-modules\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.892053 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.891959 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-sys\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.892053 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.892024 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-podres\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.892053 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.892027 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-proc\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.892305 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.892081 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-lib-modules\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:20.899567 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:20.899542 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b4jb\" (UniqueName: \"kubernetes.io/projected/c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c-kube-api-access-6b4jb\") pod \"perf-node-gather-daemonset-mscm8\" (UID: \"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c\") " pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:21.012569 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:21.012541 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:21.152632 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:21.152599 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8"] Jul 22 20:51:21.154241 ip-10-0-139-68 kubenswrapper[2553]: W0722 20:51:21.154210 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc2a5bed6_a76e_4ac6_8abd_b453a7a7db8c.slice/crio-5169dabac55c848b74a5d001ea954cc776770f8e14fbca17683eb3ad5a2d6687 WatchSource:0}: Error finding container 5169dabac55c848b74a5d001ea954cc776770f8e14fbca17683eb3ad5a2d6687: Status 404 returned error can't find the container with id 5169dabac55c848b74a5d001ea954cc776770f8e14fbca17683eb3ad5a2d6687 Jul 22 20:51:21.211002 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:21.210973 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-r7mrz_b0f4e343-fae7-43f2-9cd7-15ea347781ad/networking-console-plugin/0.log" Jul 22 20:51:21.762107 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:21.762070 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:51:21.770619 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:21.770588 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/2.log" Jul 22 20:51:22.095500 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:22.095364 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" event={"ID":"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c","Type":"ContainerStarted","Data":"20c4fe9e87f1040e61b939e965a64ed3ae6f137c512204c505e6f973f73c374b"} Jul 22 20:51:22.095652 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:22.095515 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:22.095652 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:22.095546 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" event={"ID":"c2a5bed6-a76e-4ac6-8abd-b453a7a7db8c","Type":"ContainerStarted","Data":"5169dabac55c848b74a5d001ea954cc776770f8e14fbca17683eb3ad5a2d6687"} Jul 22 20:51:22.112604 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:22.112552 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" podStartSLOduration=2.112536923 podStartE2EDuration="2.112536923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:51:22.110509145 +0000 UTC m=+2991.667252854" watchObservedRunningTime="2026-07-22 20:51:22.112536923 +0000 UTC m=+2991.669280660" Jul 22 20:51:22.233089 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:22.233060 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5bc6b4fb58-6czj2_40665578-fee1-4407-bfae-3567534ba3c2/console/0.log" Jul 22 20:51:22.272520 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:22.272489 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-xhjxq_ca4b953d-3730-4b93-b12b-feb323a8b179/download-server/0.log" Jul 22 20:51:22.827436 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:22.827389 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-d7x7d_00ec5360-2150-4da1-bd5f-d2494479bc7e/volume-data-source-validator/0.log" Jul 22 20:51:23.788065 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:23.788028 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hqpcf_c5545d45-8b89-4f48-813c-6c73a905c56a/dns/0.log" Jul 22 20:51:23.810758 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:23.810726 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hqpcf_c5545d45-8b89-4f48-813c-6c73a905c56a/kube-rbac-proxy/0.log" Jul 22 20:51:23.971665 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:23.971629 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zq625_4da17f54-06b4-4750-9f1b-3304f6686447/dns-node-resolver/0.log" Jul 22 20:51:24.601866 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:24.601834 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rbd5p_ad9034e0-a2b0-4bb7-bef5-b124b545dc59/node-ca/0.log" Jul 22 20:51:25.848400 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:25.848371 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-787b995ffc-vfsp7_ad8de055-bd52-4ba9-8b83-ac0845649cee/kube-auth-proxy/0.log" Jul 22 20:51:25.895939 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:25.895911 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qjgnt_ae4f0dfa-c828-4b82-bd55-ea4ef01369b8/istio-proxy/0.log" Jul 22 20:51:25.948023 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:25.947993 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qjgnt_ae4f0dfa-c828-4b82-bd55-ea4ef01369b8/istio-proxy/1.log" Jul 22 20:51:25.986040 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:25.986000 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-5qbzv_072bf681-5686-470f-8e3f-f13659a49b9d/payload-pre-processing/0.log" Jul 22 20:51:26.023932 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:26.023903 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-g47jq_2eb8d9e2-7b38-4512-8920-e1be5129916d/payload-processing/0.log" Jul 22 20:51:26.744877 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:26.744844 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5g5zk_8c525512-dbbd-4e74-8b1e-28270cfb6554/serve-healthcheck-canary/0.log" Jul 22 20:51:27.248975 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:27.248914 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-sfgrv_5f315bbc-8593-4ead-9e7e-06b55dee57c5/insights-operator/0.log" Jul 22 20:51:27.249501 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:27.249457 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-sfgrv_5f315bbc-8593-4ead-9e7e-06b55dee57c5/insights-operator/1.log" Jul 22 20:51:27.420700 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:27.420672 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zd8zj_0458ea69-7f6c-4806-a27c-adaf0d15ad76/kube-rbac-proxy/0.log" Jul 22 20:51:27.445488 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:27.445453 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zd8zj_0458ea69-7f6c-4806-a27c-adaf0d15ad76/exporter/0.log" Jul 22 20:51:27.469012 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:27.468982 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zd8zj_0458ea69-7f6c-4806-a27c-adaf0d15ad76/extractor/0.log" Jul 22 20:51:28.111481 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:28.111451 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nzwv7/perf-node-gather-daemonset-mscm8" Jul 22 20:51:29.569238 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:29.569203 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vn5kw_e5b35131-90f5-4246-9ba5-9c72cb136c00/manager/0.log" Jul 22 20:51:29.890809 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:29.890734 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2wrrs_dead0468-003c-4f71-8848-5b79e506e482/manager/0.log" Jul 22 20:51:30.119074 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:30.119044 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dw6vc_d11451bb-b7fa-4604-887e-8c16f46c73c1/server/0.log" Jul 22 20:51:30.236649 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:30.236618 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-857jb_85233d48-b20f-4981-a4ca-19d228a4e537/manager/0.log" Jul 22 20:51:30.449125 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:30.449099 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc56f4f-zwzkl_f0b1a600-adef-40bf-b36f-e99c5f0d9db5/manager/0.log" Jul 22 20:51:30.476395 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:30.476369 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9fc56f4f-zwzkl_f0b1a600-adef-40bf-b36f-e99c5f0d9db5/copy-manifests/0.log" Jul 22 20:51:31.284676 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:31.284651 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:51:31.299542 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:31.299518 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-p5zmg_52d3ecf9-455b-427d-9759-ecfeace7e339/console-operator/1.log" Jul 22 20:51:31.849502 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:31.849465 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-787b995ffc-vfsp7_ad8de055-bd52-4ba9-8b83-ac0845649cee/kube-auth-proxy/0.log" Jul 22 20:51:31.872175 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:31.872142 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qjgnt_ae4f0dfa-c828-4b82-bd55-ea4ef01369b8/istio-proxy/0.log" Jul 22 20:51:31.929350 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:31.929319 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qjgnt_ae4f0dfa-c828-4b82-bd55-ea4ef01369b8/istio-proxy/1.log" Jul 22 20:51:31.963905 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:31.963866 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-5qbzv_072bf681-5686-470f-8e3f-f13659a49b9d/payload-pre-processing/0.log" Jul 22 20:51:31.994033 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:31.993938 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-g47jq_2eb8d9e2-7b38-4512-8920-e1be5129916d/payload-processing/0.log" Jul 22 20:51:33.418297 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:33.418251 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-787b995ffc-vfsp7_ad8de055-bd52-4ba9-8b83-ac0845649cee/kube-auth-proxy/0.log" Jul 22 20:51:33.447744 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:33.447701 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qjgnt_ae4f0dfa-c828-4b82-bd55-ea4ef01369b8/istio-proxy/0.log" Jul 22 20:51:33.499006 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:33.498941 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qjgnt_ae4f0dfa-c828-4b82-bd55-ea4ef01369b8/istio-proxy/1.log" Jul 22 20:51:33.530022 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:33.529922 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-5qbzv_072bf681-5686-470f-8e3f-f13659a49b9d/payload-pre-processing/0.log" Jul 22 20:51:33.558107 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:33.558077 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-g47jq_2eb8d9e2-7b38-4512-8920-e1be5129916d/payload-processing/0.log" Jul 22 20:51:36.868802 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:36.868767 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-nf56t_4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc/migrator/0.log" Jul 22 20:51:36.888386 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:36.888344 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-nf56t_4b5f2c58-24d0-4ba1-a4f3-eaa4a132ccfc/graceful-termination/0.log" Jul 22 20:51:38.698623 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:38.698598 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ss5ld_e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f/kube-multus-additional-cni-plugins/0.log" Jul 22 20:51:38.719589 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:38.719564 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ss5ld_e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f/egress-router-binary-copy/0.log" Jul 22 20:51:38.747682 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:38.747657 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ss5ld_e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f/cni-plugins/0.log" Jul 22 20:51:38.772080 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:38.772056 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ss5ld_e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f/bond-cni-plugin/0.log" Jul 22 20:51:38.799644 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:38.799616 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ss5ld_e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f/routeoverride-cni/0.log" Jul 22 20:51:38.835771 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:38.835746 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ss5ld_e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f/whereabouts-cni-bincopy/0.log" Jul 22 20:51:38.868893 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:38.868862 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ss5ld_e369ae6e-7aa0-4b1e-b1ad-ccac4c3d868f/whereabouts-cni/0.log" Jul 22 20:51:38.969529 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:38.969484 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-msxvm_e0704b41-003f-4d1f-9aed-c610af2ddaae/kube-multus/0.log" Jul 22 20:51:39.056232 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:39.056203 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4cnsx_b5419f3c-d6a5-4530-872d-08505f1c1c74/network-metrics-daemon/0.log" Jul 22 20:51:39.078960 ip-10-0-139-68 kubenswrapper[2553]: I0722 20:51:39.078920 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4cnsx_b5419f3c-d6a5-4530-872d-08505f1c1c74/kube-rbac-proxy/0.log"