Jul 12 22:08:38.836478 ip-10-0-134-44 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 12 22:08:38.836490 ip-10-0-134-44 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 12 22:08:38.836497 ip-10-0-134-44 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 12 22:08:38.836737 ip-10-0-134-44 systemd[1]: Failed to start Kubernetes Kubelet. Jul 12 22:08:48.274222 ip-10-0-134-44 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 12 22:08:48.274238 ip-10-0-134-44 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ce4159e597ee41bdb39d403a964945b8 -- Jul 12 22:10:16.947540 ip-10-0-134-44 systemd[1]: Starting Kubernetes Kubelet... Jul 12 22:10:17.413549 ip-10-0-134-44 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 22:10:17.413549 ip-10-0-134-44 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 22:10:17.413549 ip-10-0-134-44 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 22:10:17.413549 ip-10-0-134-44 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 22:10:17.413549 ip-10-0-134-44 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 22:10:17.415506 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.415356 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 22:10:17.419492 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419477 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:10:17.419492 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419492 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419495 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419499 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419502 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419506 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419508 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419511 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419515 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419518 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419521 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419524 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419526 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419529 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419532 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419535 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419537 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419540 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419542 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419545 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419553 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:10:17.419559 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419556 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419558 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419561 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419563 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419566 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419569 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419572 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419575 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419578 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419580 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419583 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419587 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419590 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419593 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419595 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419598 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419601 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419603 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419605 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:10:17.420020 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419608 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419610 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419613 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419615 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419618 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419621 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419624 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419626 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419628 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419631 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419633 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419636 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419638 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419640 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419643 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419647 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419651 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419654 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419657 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:10:17.420488 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419660 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419663 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419666 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419668 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419671 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419674 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419676 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419679 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419681 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419684 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419687 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419690 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419693 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419695 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419698 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419700 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419703 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419705 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419708 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419710 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:10:17.420951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419712 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419715 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419719 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419723 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419726 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419729 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419732 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419734 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419738 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419741 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419744 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419746 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419749 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419752 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419754 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419756 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419759 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419761 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419764 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:10:17.422089 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.419766 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421637 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421644 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421649 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421654 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421657 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421660 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421663 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421666 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421669 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421672 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421675 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421678 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421681 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421684 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421699 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421702 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421705 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421707 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421710 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:10:17.422573 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421713 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421717 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421719 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421722 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421725 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421727 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421730 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421733 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421735 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421738 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421740 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421743 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421745 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421748 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421750 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421752 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421755 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421757 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421760 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:10:17.423067 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421763 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421767 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421769 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421772 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421774 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421777 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421779 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421783 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421787 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421790 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421793 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421796 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421799 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421802 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421808 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421811 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421814 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421817 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421820 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421822 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:10:17.423547 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421825 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421827 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421830 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421832 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421837 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421840 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421843 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421846 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421848 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421851 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421854 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421856 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421859 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421862 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421864 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421867 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421869 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421871 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421874 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421877 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:10:17.424039 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421879 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421881 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421884 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421886 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421888 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421891 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421895 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421897 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421900 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421903 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421905 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421908 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421910 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421913 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421915 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421918 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421920 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421923 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421925 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421928 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:10:17.424564 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.421930 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422319 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422325 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422328 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422332 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422335 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422337 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422340 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422343 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422345 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422348 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422350 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422353 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422355 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422358 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422369 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422374 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422376 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422380 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:10:17.425042 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422383 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422386 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422389 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422392 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422395 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422397 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422400 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422402 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422405 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422407 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422410 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422412 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422415 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422417 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422420 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422423 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422425 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422428 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422430 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422433 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:10:17.425504 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422436 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422438 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422441 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422443 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422446 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422448 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422450 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422453 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422455 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422457 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422460 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422465 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422467 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422470 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422472 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422475 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422478 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422480 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422483 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422485 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:10:17.426022 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422488 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422490 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422493 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422495 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422498 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422501 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422503 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422506 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422508 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422511 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422513 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422516 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422518 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422521 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422523 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422526 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422528 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422531 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422533 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422536 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:10:17.426528 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422538 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422540 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422543 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422547 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422551 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422555 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422559 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422563 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422566 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422568 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422571 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422573 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422576 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422578 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422581 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422584 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422586 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422589 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422591 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:10:17.427005 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422594 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.422596 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422673 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422683 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422692 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422698 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422705 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422710 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422716 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422721 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422724 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422727 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422731 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422734 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422737 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422740 2563 flags.go:64] FLAG: --cgroup-root="" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422744 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422747 2563 flags.go:64] FLAG: --client-ca-file="" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422750 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422753 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422758 2563 flags.go:64] FLAG: --cluster-domain="" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422760 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422764 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422767 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 22:10:17.427534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422770 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422773 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422776 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422779 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422782 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422785 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422788 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422791 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422794 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422797 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422801 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422804 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422807 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422809 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422812 2563 flags.go:64] FLAG: --enable-server="true" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422815 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422820 2563 flags.go:64] FLAG: --event-burst="100" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422823 2563 flags.go:64] FLAG: --event-qps="50" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422826 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422828 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422831 2563 flags.go:64] FLAG: --eviction-hard="" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422835 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422838 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422841 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422850 2563 flags.go:64] FLAG: --eviction-soft="" Jul 12 22:10:17.428201 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422852 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422855 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422858 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422861 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422864 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422867 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422869 2563 flags.go:64] FLAG: --feature-gates="" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422873 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422876 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422879 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422882 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422885 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422888 2563 flags.go:64] FLAG: --help="false" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422892 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422895 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422897 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422900 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422904 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422907 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422910 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422913 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422915 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422919 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 22:10:17.428810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422922 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422925 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422928 2563 flags.go:64] FLAG: --kube-reserved="" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422931 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422934 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422937 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422939 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422942 2563 flags.go:64] FLAG: --lock-file="" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422947 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422950 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422953 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422958 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422961 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422964 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422966 2563 flags.go:64] FLAG: --logging-format="text" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422969 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422972 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422975 2563 flags.go:64] FLAG: --manifest-url="" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422978 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422982 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422985 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422989 2563 flags.go:64] FLAG: --max-pods="110" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422992 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422995 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.422997 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 22:10:17.429395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423000 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423003 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423006 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423009 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423016 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423019 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423021 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423024 2563 flags.go:64] FLAG: --pod-cidr="" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423028 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423033 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423036 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423038 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423041 2563 flags.go:64] FLAG: --port="10250" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423045 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423047 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0837c3baeb70a8099" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423052 2563 flags.go:64] FLAG: --qos-reserved="" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423057 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423060 2563 flags.go:64] FLAG: --register-node="true" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423063 2563 flags.go:64] FLAG: --register-with-taints="" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423066 2563 flags.go:64] FLAG: --registry-burst="10" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423069 2563 flags.go:64] FLAG: --registry-qps="5" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423072 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423075 2563 flags.go:64] FLAG: --reserved-memory="" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423079 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423082 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 22:10:17.429991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423085 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423088 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423090 2563 flags.go:64] FLAG: --runonce="false" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423093 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423097 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423100 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423103 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423106 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423108 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423112 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423114 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423117 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423120 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423122 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423125 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423128 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423132 2563 flags.go:64] FLAG: --system-cgroups="" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423135 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423140 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423142 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423145 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423149 2563 flags.go:64] FLAG: --tls-min-version="" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423153 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423157 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423160 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 22:10:17.430653 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423162 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423165 2563 flags.go:64] FLAG: --v="2" Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423169 2563 flags.go:64] FLAG: --version="false" Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423173 2563 flags.go:64] FLAG: --vmodule="" Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423177 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423180 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423326 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423331 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423334 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423337 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423340 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423343 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423345 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423348 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423351 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423353 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423356 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423359 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423361 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423364 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423367 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:10:17.431332 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423369 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423372 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423374 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423384 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423386 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423389 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423392 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423394 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423399 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423402 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423405 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423408 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423410 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423413 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423415 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423419 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423423 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423426 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423428 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423431 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:10:17.431858 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423434 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423437 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423439 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423442 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423445 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423447 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423450 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423452 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423455 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423458 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423460 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423463 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423465 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423469 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423472 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423475 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423478 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423481 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423483 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423486 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:10:17.432358 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423489 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423493 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423496 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423498 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423501 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423503 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423505 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423509 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423513 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423516 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423519 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423522 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423524 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423527 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423530 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423533 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423535 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423537 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423540 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:10:17.432894 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423542 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423545 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423547 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423550 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423552 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423555 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423558 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423560 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423563 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423566 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423571 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423573 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423576 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423579 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423584 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423586 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423589 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423591 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423594 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423596 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:10:17.433380 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423599 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423601 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423603 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423606 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.423609 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.423614 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.430426 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.430440 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.430460 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.430466 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430520 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430525 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430529 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430532 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430535 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:10:17.434032 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430538 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430541 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430543 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430546 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430549 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430551 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430554 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430556 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430559 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430561 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430564 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430566 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430569 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430571 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430574 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430576 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430580 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430584 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430587 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430590 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:10:17.434553 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430593 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430595 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430598 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430601 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430605 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430608 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430610 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430613 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430615 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430618 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430620 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430623 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430625 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430628 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430631 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430633 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430636 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430638 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430641 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430643 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:10:17.435061 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430646 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430648 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430650 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430653 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430656 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430658 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430661 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430663 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430666 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430669 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430671 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430674 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430677 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430680 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430682 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430685 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430688 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430691 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430693 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430696 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:10:17.435595 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430698 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430701 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430703 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430706 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430708 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430710 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430713 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430715 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430717 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430720 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430722 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430724 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430727 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430729 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430731 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430736 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430739 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430742 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430744 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:10:17.436132 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430747 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430750 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430752 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430755 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430757 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430760 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430762 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430765 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430767 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430770 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430772 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430775 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430778 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430782 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:10:17.436714 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430785 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.430790 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430900 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430905 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430908 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430911 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430914 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430917 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430919 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430922 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430924 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430927 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430929 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430931 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430934 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 22:10:17.437081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430936 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430939 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430941 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430944 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430946 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430948 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430951 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430953 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430956 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430958 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430960 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430963 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430965 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430968 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430970 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430973 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430976 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430978 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430980 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430983 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 22:10:17.437462 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430985 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430987 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430990 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430992 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430994 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430997 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.430999 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431001 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431004 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431006 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431009 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431011 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431014 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431016 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431018 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431021 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431023 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431026 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431028 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 22:10:17.437951 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431031 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431033 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431035 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431038 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431040 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431043 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431045 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431047 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431050 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431053 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431055 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431058 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431060 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431062 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431065 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431067 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431069 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431072 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431074 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431076 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 22:10:17.438552 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431079 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431081 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431083 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431086 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431088 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431090 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431094 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431096 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431099 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431102 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431106 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431108 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431111 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431113 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431116 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431118 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431120 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431123 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431125 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 22:10:17.439378 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431128 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431130 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431132 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431135 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431139 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431142 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431145 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:17.431149 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.431154 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.431247 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.434380 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.435897 2563 server.go:1019] "Starting client certificate rotation" Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.435998 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 22:10:17.440164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.436032 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 22:10:17.440796 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.438946 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 22:10:17.440796 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.440701 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 22:10:17.450221 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.450201 2563 log.go:25] "Validated CRI v1 runtime API" Jul 12 22:10:17.452425 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.452411 2563 log.go:25] "Validated CRI v1 image API" Jul 12 22:10:17.453228 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.453212 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 22:10:17.455738 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.455714 2563 fs.go:135] Filesystem UUIDs: map[0f3ff86b-d99e-4520-9a10-53ad9f3082b6:/dev/nvme0n1p3 58064ce5-ea9d-4557-a739-f1cccfc4f8e8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 12 22:10:17.455797 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.455739 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 22:10:17.460557 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.460441 2563 manager.go:216] Machine: {Timestamp:2026-07-12 22:10:17.459943733 +0000 UTC m=+0.381657824 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3112536 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bf800643c6ca4f4c8eeac6d01b773 SystemUUID:ec2bf800-643c-6ca4-f4c8-eeac6d01b773 BootID:ce4159e5-97ee-41bd-b39d-403a964945b8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:33:54:c8:a9:d7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:33:54:c8:a9:d7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:17:f1:5a:cb:b0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 22:10:17.460557 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.460550 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 22:10:17.460677 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.460620 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 22:10:17.463499 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.463472 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 22:10:17.463621 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.463501 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-44.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 22:10:17.464244 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.464235 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 22:10:17.464290 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.464247 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 22:10:17.464290 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.464254 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 22:10:17.464290 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.464275 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 22:10:17.465470 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.465461 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 22:10:17.472388 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.472373 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 22:10:17.472644 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.472633 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 12 22:10:17.472761 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.472751 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 22:10:17.472804 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.472781 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 12 22:10:17.472804 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.472800 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 22:10:17.472889 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.472817 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 22:10:17.472889 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.472826 2563 kubelet.go:398] "Adding apiserver pod source" Jul 12 22:10:17.472889 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.472834 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 22:10:17.473159 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.473148 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 22:10:17.473195 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.473166 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 22:10:17.477800 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.477780 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 22:10:17.479652 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.479635 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 22:10:17.479739 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.479674 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 22:10:17.479878 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.479864 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 22:10:17.479942 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.479884 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 22:10:17.479942 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.479895 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 22:10:17.479942 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.479904 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 22:10:17.479942 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.479916 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 22:10:17.479942 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.479925 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 22:10:17.480373 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.480362 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 22:10:17.480461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.480376 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 22:10:17.480461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.480387 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 22:10:17.480461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.480396 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 22:10:17.480461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.480410 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 22:10:17.480461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.480423 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 22:10:17.481105 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.481095 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 22:10:17.481155 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.481108 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 22:10:17.484056 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.484031 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-44.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 22:10:17.484122 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.484075 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 22:10:17.484179 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.484162 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-44.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 22:10:17.484357 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.484344 2563 server.go:1268] "Started kubelet" Jul 12 22:10:17.484428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.484402 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 22:10:17.484487 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.484436 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 22:10:17.484539 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.484511 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 22:10:17.485199 ip-10-0-134-44 systemd[1]: Started Kubernetes Kubelet. Jul 12 22:10:17.487419 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.487401 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 12 22:10:17.488530 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.488509 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 22:10:17.490111 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.490094 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 22:10:17.493426 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.493369 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 22:10:17.493426 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.493379 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 22:10:17.493577 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.493461 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:17.494209 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.494190 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 22:10:17.494311 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.494196 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 22:10:17.494311 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.494225 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 22:10:17.494311 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.493218 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-44.ec2.internal.18c1aa56de06b863 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-44.ec2.internal,UID:ip-10-0-134-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-44.ec2.internal,},FirstTimestamp:2026-07-12 22:10:17.484318819 +0000 UTC m=+0.406032889,LastTimestamp:2026-07-12 22:10:17.484318819 +0000 UTC m=+0.406032889,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-44.ec2.internal,}" Jul 12 22:10:17.494459 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.494335 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 12 22:10:17.494459 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.494341 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 12 22:10:17.494904 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.494891 2563 factory.go:55] Registering systemd factory Jul 12 22:10:17.494904 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.494907 2563 factory.go:223] Registration of the systemd container factory successfully Jul 12 22:10:17.495098 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.495085 2563 factory.go:153] Registering CRI-O factory Jul 12 22:10:17.495149 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.495102 2563 factory.go:223] Registration of the crio container factory successfully Jul 12 22:10:17.495194 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.495160 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 22:10:17.495194 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.495189 2563 factory.go:103] Registering Raw factory Jul 12 22:10:17.495300 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.495204 2563 manager.go:1196] Started watching for new ooms in manager Jul 12 22:10:17.495854 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.495839 2563 manager.go:349] Starting recovery of all containers Jul 12 22:10:17.496161 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.496136 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 12 22:10:17.496243 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.496187 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-44.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 12 22:10:17.505690 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.505385 2563 manager.go:354] Recovery completed Jul 12 22:10:17.507351 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.507322 2563 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 12 22:10:17.510400 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.510386 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:10:17.512119 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.512090 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:10:17.512187 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.512132 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:10:17.512187 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.512143 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:10:17.512594 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.512580 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 22:10:17.512594 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.512591 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:10:17.512718 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.512594 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 22:10:17.512718 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.512635 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 12 22:10:17.513232 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.513220 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:10:17.513331 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.513244 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:10:17.513331 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.513254 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:10:17.513811 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.513798 2563 policy_none.go:49] "None policy: Start" Jul 12 22:10:17.513883 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.513814 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 22:10:17.513883 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.513826 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 22:10:17.513981 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.513885 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-44.ec2.internal.18c1aa56dfaeebd7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-44.ec2.internal,UID:ip-10-0-134-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-44.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-44.ec2.internal,},FirstTimestamp:2026-07-12 22:10:17.512119255 +0000 UTC m=+0.433833325,LastTimestamp:2026-07-12 22:10:17.512119255 +0000 UTC m=+0.433833325,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-44.ec2.internal,}" Jul 12 22:10:17.514726 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.514716 2563 policy_none.go:47] "Start" Jul 12 22:10:17.516121 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.516105 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cmmwt" Jul 12 22:10:17.522916 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.522851 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-44.ec2.internal.18c1aa56dfaf32fd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-44.ec2.internal,UID:ip-10-0-134-44.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-44.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-44.ec2.internal,},FirstTimestamp:2026-07-12 22:10:17.512137469 +0000 UTC m=+0.433851543,LastTimestamp:2026-07-12 22:10:17.512137469 +0000 UTC m=+0.433851543,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-44.ec2.internal,}" Jul 12 22:10:17.523761 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.523741 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cmmwt" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.556745 2563 manager.go:337] "Starting Device Plugin manager" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.556778 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.556791 2563 server.go:87] "Starting device plugin registration server" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.557031 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.557043 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.557122 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.557184 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.557190 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.557761 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.557811 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.570149 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.571459 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.571490 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.571509 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.571538 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 22:10:17.574709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.573617 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:10:17.657930 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.657909 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:10:17.658895 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.658881 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:10:17.658985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.658906 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:10:17.658985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.658917 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:10:17.658985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.658944 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.667707 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.667670 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.667707 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.667691 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-44.ec2.internal\": node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:17.671957 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.671912 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal"] Jul 12 22:10:17.672010 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.672001 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:10:17.672746 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.672726 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:10:17.672827 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.672760 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:10:17.672827 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.672772 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:10:17.673811 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.673799 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:10:17.673918 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.673894 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.673918 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.673921 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:10:17.674617 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.674598 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:10:17.674617 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.674611 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:10:17.674735 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.674633 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:10:17.674735 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.674639 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:10:17.674735 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.674643 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:10:17.674735 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.674649 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:10:17.675642 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.675626 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.675694 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.675650 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:10:17.676245 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.676228 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:10:17.676349 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.676259 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:10:17.676349 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.676298 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:10:17.695749 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.695726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73621ab645bf612f0f90ffad1d6de781-config\") pod \"kube-apiserver-proxy-ip-10-0-134-44.ec2.internal\" (UID: \"73621ab645bf612f0f90ffad1d6de781\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.695835 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.695755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/864f4becf45250c71ccc80209b0b9c7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal\" (UID: \"864f4becf45250c71ccc80209b0b9c7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.695835 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.695781 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/864f4becf45250c71ccc80209b0b9c7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal\" (UID: \"864f4becf45250c71ccc80209b0b9c7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.696168 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.696153 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:17.704924 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.704906 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-44.ec2.internal\" not found" node="ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.709543 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.709530 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-44.ec2.internal\" not found" node="ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.795911 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.795874 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/864f4becf45250c71ccc80209b0b9c7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal\" (UID: \"864f4becf45250c71ccc80209b0b9c7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.795911 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.795913 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/864f4becf45250c71ccc80209b0b9c7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal\" (UID: \"864f4becf45250c71ccc80209b0b9c7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.796059 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.795931 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73621ab645bf612f0f90ffad1d6de781-config\") pod \"kube-apiserver-proxy-ip-10-0-134-44.ec2.internal\" (UID: \"73621ab645bf612f0f90ffad1d6de781\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.796059 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.795957 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73621ab645bf612f0f90ffad1d6de781-config\") pod \"kube-apiserver-proxy-ip-10-0-134-44.ec2.internal\" (UID: \"73621ab645bf612f0f90ffad1d6de781\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.796059 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.795964 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/864f4becf45250c71ccc80209b0b9c7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal\" (UID: \"864f4becf45250c71ccc80209b0b9c7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.796059 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:17.795970 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/864f4becf45250c71ccc80209b0b9c7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal\" (UID: \"864f4becf45250c71ccc80209b0b9c7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" Jul 12 22:10:17.796927 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.796910 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:17.897575 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.897541 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:17.998368 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:17.998295 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:18.007485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.007455 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" Jul 12 22:10:18.012413 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.012395 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal" Jul 12 22:10:18.099357 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:18.099321 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:18.199766 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:18.199739 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:18.300329 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:18.300240 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:18.377930 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.377893 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:10:18.400464 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:18.400435 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:18.435626 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.435593 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 22:10:18.436092 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.435724 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 22:10:18.436092 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.435729 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 22:10:18.501330 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:18.501297 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:18.526625 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.526582 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 22:05:18 +0000 UTC" deadline="2028-01-17 23:52:27.364700908 +0000 UTC" Jul 12 22:10:18.526625 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.526619 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13297h42m8.838085326s" Jul 12 22:10:18.560070 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:18.560043 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73621ab645bf612f0f90ffad1d6de781.slice/crio-1ceef4c967f12f980eb603b76fbbaee88f383598097071c8ef8124e956159601 WatchSource:0}: Error finding container 1ceef4c967f12f980eb603b76fbbaee88f383598097071c8ef8124e956159601: Status 404 returned error can't find the container with id 1ceef4c967f12f980eb603b76fbbaee88f383598097071c8ef8124e956159601 Jul 12 22:10:18.560316 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:18.560293 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864f4becf45250c71ccc80209b0b9c7f.slice/crio-d090c8b29974a69685ad13d11fb436efef3436fe0788f1c01704503fb35e228f WatchSource:0}: Error finding container d090c8b29974a69685ad13d11fb436efef3436fe0788f1c01704503fb35e228f: Status 404 returned error can't find the container with id d090c8b29974a69685ad13d11fb436efef3436fe0788f1c01704503fb35e228f Jul 12 22:10:18.564484 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.564461 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:10:18.574483 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.574446 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal" event={"ID":"73621ab645bf612f0f90ffad1d6de781","Type":"ContainerStarted","Data":"1ceef4c967f12f980eb603b76fbbaee88f383598097071c8ef8124e956159601"} Jul 12 22:10:18.576165 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.576136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" event={"ID":"864f4becf45250c71ccc80209b0b9c7f","Type":"ContainerStarted","Data":"d090c8b29974a69685ad13d11fb436efef3436fe0788f1c01704503fb35e228f"} Jul 12 22:10:18.597148 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.597122 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 22:10:18.598137 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.598119 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientMemory" Jul 12 22:10:18.598226 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.598154 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 22:10:18.598226 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.598168 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeHasSufficientPID" Jul 12 22:10:18.601784 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:18.601769 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:18.701960 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:18.701920 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-44.ec2.internal\" not found" Jul 12 22:10:18.733437 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.733412 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:10:18.794549 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.794516 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" Jul 12 22:10:18.809832 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.809771 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 22:10:18.810989 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.810976 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal" Jul 12 22:10:18.820240 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:18.820217 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 22:10:20.078040 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.078015 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:10:20.324746 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.324653 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:10:20.527996 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.527960 2563 apiserver.go:52] "Watching apiserver" Jul 12 22:10:20.535196 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.535172 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 22:10:20.537474 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.537439 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-cd9nz","kube-system/konnectivity-agent-g7vtm","kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq","openshift-dns/node-resolver-vht46","openshift-image-registry/node-ca-7wm2n","openshift-multus/multus-qm5vb","openshift-network-operator/iptables-alerter-l9cjh","openshift-cluster-node-tuning-operator/tuned-55vd4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal","openshift-multus/multus-additional-cni-plugins-chlj5","openshift-multus/network-metrics-daemon-gq8dd","openshift-network-diagnostics/network-check-target-pzppf","openshift-ovn-kubernetes/ovnkube-node-bvgbg"] Jul 12 22:10:20.539087 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.539051 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.540342 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.540323 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:20.541574 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.541554 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:20.541659 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.541626 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:20.541989 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.541967 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 22:10:20.542133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.542118 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 22:10:20.542236 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.542118 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-69srw\"" Jul 12 22:10:20.542821 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.542649 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.542821 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.542789 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 22:10:20.543680 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.543655 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qhtpw\"" Jul 12 22:10:20.544180 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.544161 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 22:10:20.544491 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.544471 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.544919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.544901 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8k662\"" Jul 12 22:10:20.544997 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.544979 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 22:10:20.545307 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.545065 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 22:10:20.545307 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.545132 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 22:10:20.545488 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.545451 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 22:10:20.546034 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.545643 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 22:10:20.546034 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.545654 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.546745 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.546725 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 22:10:20.547180 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.547160 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 22:10:20.547418 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.547401 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pbwjv\"" Jul 12 22:10:20.547949 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.547928 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 22:10:20.548137 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.548120 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 22:10:20.550923 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.548423 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 22:10:20.550923 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.549157 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 22:10:20.551720 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.551546 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zcx4x\"" Jul 12 22:10:20.552393 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.551810 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.552393 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.551835 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.554590 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.553844 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.554590 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.554223 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 22:10:20.554728 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.554624 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pk4nv\"" Jul 12 22:10:20.555224 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.554845 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 22:10:20.555224 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.555048 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 22:10:20.560611 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560583 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-sysctl-conf\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.560711 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560628 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-sys\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.560711 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560653 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e80707e-d52d-4b32-ac55-51afc01ccf59-tmp\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.560711 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84b7f93a-2be5-4673-8f1c-9a88435b4e91-hosts-file\") pod \"node-resolver-vht46\" (UID: \"84b7f93a-2be5-4673-8f1c-9a88435b4e91\") " pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.560845 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560711 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-cnibin\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.560845 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560739 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-modprobe-d\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.560845 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560760 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-host\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.560845 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560786 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khn9g\" (UniqueName: \"kubernetes.io/projected/71466df1-3f21-4c6a-acbe-758aeea5531d-kube-api-access-khn9g\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:20.560845 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560815 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-os-release\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.561058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560843 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krwlh\" (UniqueName: \"kubernetes.io/projected/b29b61f6-9dba-468c-adcd-146cbfc66a27-kube-api-access-krwlh\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.561058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560874 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-cni-dir\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560901 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-cnibin\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560923 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-sysconfig\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.561058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560957 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-sysctl-d\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.561058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.560983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-lib-modules\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.561058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561012 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1cbd4733-347d-40b6-a124-7f8df70f0ec6-agent-certs\") pod \"konnectivity-agent-g7vtm\" (UID: \"1cbd4733-347d-40b6-a124-7f8df70f0ec6\") " pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:20.561058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561035 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-tuning-conf-dir\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.561428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561066 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ed266b7-f4d9-4768-83e8-493ee6015789-cni-binary-copy\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561094 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-run-netns\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561121 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbqtv\" (UniqueName: \"kubernetes.io/projected/84b7f93a-2be5-4673-8f1c-9a88435b4e91-kube-api-access-lbqtv\") pod \"node-resolver-vht46\" (UID: \"84b7f93a-2be5-4673-8f1c-9a88435b4e91\") " pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.561428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561149 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-var-lib-cni-bin\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561177 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-var-lib-cni-multus\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-run-multus-certs\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561228 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1cbd4733-347d-40b6-a124-7f8df70f0ec6-konnectivity-ca\") pod \"konnectivity-agent-g7vtm\" (UID: \"1cbd4733-347d-40b6-a124-7f8df70f0ec6\") " pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:20.561428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b7f93a-2be5-4673-8f1c-9a88435b4e91-tmp-dir\") pod \"node-resolver-vht46\" (UID: \"84b7f93a-2be5-4673-8f1c-9a88435b4e91\") " pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.561759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561449 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/255bf093-2970-45da-b373-4729e0e39673-host\") pod \"node-ca-7wm2n\" (UID: \"255bf093-2970-45da-b373-4729e0e39673\") " pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.561759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561483 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b29b61f6-9dba-468c-adcd-146cbfc66a27-cni-binary-copy\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.561759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561541 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-system-cni-dir\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-var-lib-kubelet\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-hostroot\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561669 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-conf-dir\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.561759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561708 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-systemd\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.562054 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561761 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-run\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.562054 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561792 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-var-lib-kubelet\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.562054 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561838 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-tuned\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.562054 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561887 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:20.562054 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-socket-dir-parent\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.562054 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561969 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-daemon-config\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.562054 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.561996 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/255bf093-2970-45da-b373-4729e0e39673-serviceca\") pod \"node-ca-7wm2n\" (UID: \"255bf093-2970-45da-b373-4729e0e39673\") " pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.562054 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562021 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b29b61f6-9dba-468c-adcd-146cbfc66a27-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.562054 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562050 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-run-k8s-cni-cncf-io\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.562485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562102 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-kubernetes\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.562485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562131 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsz9q\" (UniqueName: \"kubernetes.io/projected/4e80707e-d52d-4b32-ac55-51afc01ccf59-kube-api-access-jsz9q\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.562485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562162 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b29b61f6-9dba-468c-adcd-146cbfc66a27-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.562485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562185 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-os-release\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.562485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562212 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-etc-kubernetes\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.562485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvl55\" (UniqueName: \"kubernetes.io/projected/1ed266b7-f4d9-4768-83e8-493ee6015789-kube-api-access-wvl55\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.562485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562266 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlqpb\" (UniqueName: \"kubernetes.io/projected/255bf093-2970-45da-b373-4729e0e39673-kube-api-access-rlqpb\") pod \"node-ca-7wm2n\" (UID: \"255bf093-2970-45da-b373-4729e0e39673\") " pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.562485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562359 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-system-cni-dir\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.562485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562461 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 22:10:20.562967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562598 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 22:10:20.562967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562692 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 22:10:20.562967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562715 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pm2bv\"" Jul 12 22:10:20.562967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562852 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 22:10:20.562967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.562918 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t65pq\"" Jul 12 22:10:20.563912 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.563266 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:20.563912 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.563346 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:20.563912 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.563676 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:20.563912 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.563742 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:20.565591 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.565568 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 22:10:20.565754 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.565728 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.569933 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.569670 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 22:10:20.569933 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.569703 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8sjhs\"" Jul 12 22:10:20.569933 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.569739 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 22:10:20.569933 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.569803 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 22:10:20.569933 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.569844 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 22:10:20.569933 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.569921 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 22:10:20.570258 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.569976 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 22:10:20.580886 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.580868 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v9kdh" Jul 12 22:10:20.588924 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.588888 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v9kdh" Jul 12 22:10:20.648532 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.648504 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 22:10:20.662761 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662719 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-ovnkube-script-lib\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.662761 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcc2f\" (UniqueName: \"kubernetes.io/projected/856900cf-180b-445f-8b7d-8616d1b93228-kube-api-access-mcc2f\") pod \"iptables-alerter-l9cjh\" (UID: \"856900cf-180b-445f-8b7d-8616d1b93228\") " pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662785 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-kubernetes\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662808 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b29b61f6-9dba-468c-adcd-146cbfc66a27-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662828 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-os-release\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/856900cf-180b-445f-8b7d-8616d1b93228-iptables-alerter-script\") pod \"iptables-alerter-l9cjh\" (UID: \"856900cf-180b-445f-8b7d-8616d1b93228\") " pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662858 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-kubernetes\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662874 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-var-lib-openvswitch\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662914 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-run-ovn\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662943 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-os-release\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662945 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-system-cni-dir\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.662999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-sysctl-conf\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663027 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-sys\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e80707e-d52d-4b32-ac55-51afc01ccf59-tmp\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663072 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84b7f93a-2be5-4673-8f1c-9a88435b4e91-hosts-file\") pod \"node-resolver-vht46\" (UID: \"84b7f93a-2be5-4673-8f1c-9a88435b4e91\") " pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-cnibin\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663120 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-modprobe-d\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663125 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-sysctl-conf\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663141 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-host\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663163 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khn9g\" (UniqueName: \"kubernetes.io/projected/71466df1-3f21-4c6a-acbe-758aeea5531d-kube-api-access-khn9g\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663186 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-os-release\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663210 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-cnibin\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663241 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-var-lib-kubelet\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-hostroot\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663310 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-sysconfig\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663328 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b29b61f6-9dba-468c-adcd-146cbfc66a27-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663345 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-sysctl-d\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663393 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ed266b7-f4d9-4768-83e8-493ee6015789-cni-binary-copy\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-system-cni-dir\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663458 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-os-release\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663470 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-sysconfig\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.663668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663521 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-hostroot\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663547 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-modprobe-d\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663564 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-host\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663577 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-sysctl-d\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663589 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-var-lib-kubelet\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663607 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-var-lib-cni-multus\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663646 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84b7f93a-2be5-4673-8f1c-9a88435b4e91-hosts-file\") pod \"node-resolver-vht46\" (UID: \"84b7f93a-2be5-4673-8f1c-9a88435b4e91\") " pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663666 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-var-lib-cni-multus\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663633 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-cnibin\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663667 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-registration-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663695 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-sys\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663719 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-slash\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663747 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-run-ovn-kubernetes\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663774 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-ovn-node-metrics-cert\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-var-lib-cni-bin\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663839 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663877 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663905 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-var-lib-cni-bin\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.664361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663549 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.663984 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-systemd-units\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664011 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-cnibin\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664103 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-etc-openvswitch\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664172 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-ovnkube-config\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664225 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ed266b7-f4d9-4768-83e8-493ee6015789-cni-binary-copy\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664218 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664297 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-daemon-config\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664322 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1cbd4733-347d-40b6-a124-7f8df70f0ec6-konnectivity-ca\") pod \"konnectivity-agent-g7vtm\" (UID: \"1cbd4733-347d-40b6-a124-7f8df70f0ec6\") " pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664343 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/255bf093-2970-45da-b373-4729e0e39673-host\") pod \"node-ca-7wm2n\" (UID: \"255bf093-2970-45da-b373-4729e0e39673\") " pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664383 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b29b61f6-9dba-468c-adcd-146cbfc66a27-cni-binary-copy\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-conf-dir\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664429 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-env-overrides\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hqvw\" (UniqueName: \"kubernetes.io/projected/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-kube-api-access-9hqvw\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664472 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-run-systemd\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664509 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-node-log\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.665114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664527 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/255bf093-2970-45da-b373-4729e0e39673-serviceca\") pod \"node-ca-7wm2n\" (UID: \"255bf093-2970-45da-b373-4729e0e39673\") " pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664547 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b29b61f6-9dba-468c-adcd-146cbfc66a27-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664568 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-run-k8s-cni-cncf-io\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664587 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-socket-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsz9q\" (UniqueName: \"kubernetes.io/projected/4e80707e-d52d-4b32-ac55-51afc01ccf59-kube-api-access-jsz9q\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664629 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87116643-5000-4846-a88b-a452ab0ef89e-kubelet-config\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664649 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlqpb\" (UniqueName: \"kubernetes.io/projected/255bf093-2970-45da-b373-4729e0e39673-kube-api-access-rlqpb\") pod \"node-ca-7wm2n\" (UID: \"255bf093-2970-45da-b373-4729e0e39673\") " pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664690 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-cni-dir\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-device-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664732 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-daemon-config\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664750 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-sys-fs\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664787 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-etc-kubernetes\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664813 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvl55\" (UniqueName: \"kubernetes.io/projected/1ed266b7-f4d9-4768-83e8-493ee6015789-kube-api-access-wvl55\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664839 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krwlh\" (UniqueName: \"kubernetes.io/projected/b29b61f6-9dba-468c-adcd-146cbfc66a27-kube-api-access-krwlh\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664864 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-run-netns\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664890 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-run-netns\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664914 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-run-openvswitch\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.665919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664941 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-cni-netd\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665016 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-cni-dir\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665171 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-conf-dir\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665204 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/255bf093-2970-45da-b373-4729e0e39673-host\") pod \"node-ca-7wm2n\" (UID: \"255bf093-2970-45da-b373-4729e0e39673\") " pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665441 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1cbd4733-347d-40b6-a124-7f8df70f0ec6-konnectivity-ca\") pod \"konnectivity-agent-g7vtm\" (UID: \"1cbd4733-347d-40b6-a124-7f8df70f0ec6\") " pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.664969 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665490 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/255bf093-2970-45da-b373-4729e0e39673-serviceca\") pod \"node-ca-7wm2n\" (UID: \"255bf093-2970-45da-b373-4729e0e39673\") " pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665508 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-lib-modules\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665571 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-run-netns\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665610 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b29b61f6-9dba-468c-adcd-146cbfc66a27-cni-binary-copy\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665614 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-lib-modules\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.665653 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665650 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-run-k8s-cni-cncf-io\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.665724 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs podName:71466df1-3f21-4c6a-acbe-758aeea5531d nodeName:}" failed. No retries permitted until 2026-07-12 22:10:21.165703919 +0000 UTC m=+3.034314735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs") pod "network-metrics-daemon-gq8dd" (UID: "71466df1-3f21-4c6a-acbe-758aeea5531d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665818 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-etc-kubernetes\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665862 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1cbd4733-347d-40b6-a124-7f8df70f0ec6-agent-certs\") pod \"konnectivity-agent-g7vtm\" (UID: \"1cbd4733-347d-40b6-a124-7f8df70f0ec6\") " pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.665908 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-tuning-conf-dir\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.666753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666060 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-run-multus-certs\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666108 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtlhs\" (UniqueName: \"kubernetes.io/projected/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-kube-api-access-xtlhs\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666138 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbqtv\" (UniqueName: \"kubernetes.io/projected/84b7f93a-2be5-4673-8f1c-9a88435b4e91-kube-api-access-lbqtv\") pod \"node-resolver-vht46\" (UID: \"84b7f93a-2be5-4673-8f1c-9a88435b4e91\") " pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666201 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b29b61f6-9dba-468c-adcd-146cbfc66a27-tuning-conf-dir\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666214 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-log-socket\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666239 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87116643-5000-4846-a88b-a452ab0ef89e-dbus\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666255 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-host-run-multus-certs\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666297 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b7f93a-2be5-4673-8f1c-9a88435b4e91-tmp-dir\") pod \"node-resolver-vht46\" (UID: \"84b7f93a-2be5-4673-8f1c-9a88435b4e91\") " pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-system-cni-dir\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666502 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-kubelet\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666519 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-system-cni-dir\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666534 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-systemd\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-run\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666606 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-var-lib-kubelet\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666629 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-tuned\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666659 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-socket-dir-parent\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.667480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666682 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-systemd\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.667985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666698 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-var-lib-kubelet\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.667985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666701 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/856900cf-180b-445f-8b7d-8616d1b93228-host-slash\") pod \"iptables-alerter-l9cjh\" (UID: \"856900cf-180b-445f-8b7d-8616d1b93228\") " pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.667985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-cni-bin\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.667985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666752 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ed266b7-f4d9-4768-83e8-493ee6015789-multus-socket-dir-parent\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.667985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666776 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84b7f93a-2be5-4673-8f1c-9a88435b4e91-tmp-dir\") pod \"node-resolver-vht46\" (UID: \"84b7f93a-2be5-4673-8f1c-9a88435b4e91\") " pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.667985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.666793 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e80707e-d52d-4b32-ac55-51afc01ccf59-run\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.667985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.667070 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b29b61f6-9dba-468c-adcd-146cbfc66a27-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.667985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.667629 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e80707e-d52d-4b32-ac55-51afc01ccf59-tmp\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.668927 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.668903 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e80707e-d52d-4b32-ac55-51afc01ccf59-etc-tuned\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.669263 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.669237 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1cbd4733-347d-40b6-a124-7f8df70f0ec6-agent-certs\") pod \"konnectivity-agent-g7vtm\" (UID: \"1cbd4733-347d-40b6-a124-7f8df70f0ec6\") " pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:20.673044 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.673021 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlqpb\" (UniqueName: \"kubernetes.io/projected/255bf093-2970-45da-b373-4729e0e39673-kube-api-access-rlqpb\") pod \"node-ca-7wm2n\" (UID: \"255bf093-2970-45da-b373-4729e0e39673\") " pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.673491 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.673467 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khn9g\" (UniqueName: \"kubernetes.io/projected/71466df1-3f21-4c6a-acbe-758aeea5531d-kube-api-access-khn9g\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:20.674335 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.674317 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsz9q\" (UniqueName: \"kubernetes.io/projected/4e80707e-d52d-4b32-ac55-51afc01ccf59-kube-api-access-jsz9q\") pod \"tuned-55vd4\" (UID: \"4e80707e-d52d-4b32-ac55-51afc01ccf59\") " pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.674756 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.674712 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krwlh\" (UniqueName: \"kubernetes.io/projected/b29b61f6-9dba-468c-adcd-146cbfc66a27-kube-api-access-krwlh\") pod \"multus-additional-cni-plugins-chlj5\" (UID: \"b29b61f6-9dba-468c-adcd-146cbfc66a27\") " pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.674965 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.674946 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvl55\" (UniqueName: \"kubernetes.io/projected/1ed266b7-f4d9-4768-83e8-493ee6015789-kube-api-access-wvl55\") pod \"multus-qm5vb\" (UID: \"1ed266b7-f4d9-4768-83e8-493ee6015789\") " pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.675056 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.675027 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbqtv\" (UniqueName: \"kubernetes.io/projected/84b7f93a-2be5-4673-8f1c-9a88435b4e91-kube-api-access-lbqtv\") pod \"node-resolver-vht46\" (UID: \"84b7f93a-2be5-4673-8f1c-9a88435b4e91\") " pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.767417 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-registration-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.767582 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767427 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-slash\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767582 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767456 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-run-ovn-kubernetes\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767582 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767489 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-ovn-node-metrics-cert\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767582 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767513 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.767582 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767517 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-registration-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.767582 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767536 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.767582 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767559 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-run-ovn-kubernetes\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767574 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767599 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-systemd-units\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767620 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-etc-openvswitch\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767636 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-systemd-units\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767517 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-slash\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767635 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-ovnkube-config\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767680 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-etc-openvswitch\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767649 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767719 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-env-overrides\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767743 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hqvw\" (UniqueName: \"kubernetes.io/projected/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-kube-api-access-9hqvw\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767780 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-run-systemd\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767802 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-node-log\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767828 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-socket-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767834 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-run-systemd\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.767804 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:20.767875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767852 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87116643-5000-4846-a88b-a452ab0ef89e-kubelet-config\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767901 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-device-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.767911 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret podName:87116643-5000-4846-a88b-a452ab0ef89e nodeName:}" failed. No retries permitted until 2026-07-12 22:10:21.267889036 +0000 UTC m=+3.136499852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret") pod "global-pull-secret-syncer-cd9nz" (UID: "87116643-5000-4846-a88b-a452ab0ef89e") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767938 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-sys-fs\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767947 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-device-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767967 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-run-netns\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767979 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-socket-dir\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767991 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87116643-5000-4846-a88b-a452ab0ef89e-kubelet-config\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768021 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-run-netns\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.767991 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-run-openvswitch\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768033 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-node-log\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768036 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-sys-fs\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-cni-netd\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768064 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-cni-netd\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768097 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-run-openvswitch\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.768759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtlhs\" (UniqueName: \"kubernetes.io/projected/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-kube-api-access-xtlhs\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-log-socket\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768159 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768172 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-env-overrides\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87116643-5000-4846-a88b-a452ab0ef89e-dbus\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768206 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-ovnkube-config\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768213 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-log-socket\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768235 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-kubelet\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/856900cf-180b-445f-8b7d-8616d1b93228-host-slash\") pod \"iptables-alerter-l9cjh\" (UID: \"856900cf-180b-445f-8b7d-8616d1b93228\") " pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768289 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-cni-bin\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-ovnkube-script-lib\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768317 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-kubelet\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768321 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/856900cf-180b-445f-8b7d-8616d1b93228-host-slash\") pod \"iptables-alerter-l9cjh\" (UID: \"856900cf-180b-445f-8b7d-8616d1b93228\") " pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768337 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcc2f\" (UniqueName: \"kubernetes.io/projected/856900cf-180b-445f-8b7d-8616d1b93228-kube-api-access-mcc2f\") pod \"iptables-alerter-l9cjh\" (UID: \"856900cf-180b-445f-8b7d-8616d1b93228\") " pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768349 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-host-cni-bin\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768395 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/856900cf-180b-445f-8b7d-8616d1b93228-iptables-alerter-script\") pod \"iptables-alerter-l9cjh\" (UID: \"856900cf-180b-445f-8b7d-8616d1b93228\") " pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.769352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768403 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87116643-5000-4846-a88b-a452ab0ef89e-dbus\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:20.769939 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-var-lib-openvswitch\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769939 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768437 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-run-ovn\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769939 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768482 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-run-ovn\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769939 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768478 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-var-lib-openvswitch\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769939 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768833 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-ovnkube-script-lib\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.769939 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.768876 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/856900cf-180b-445f-8b7d-8616d1b93228-iptables-alerter-script\") pod \"iptables-alerter-l9cjh\" (UID: \"856900cf-180b-445f-8b7d-8616d1b93228\") " pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.770711 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.770692 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-ovn-node-metrics-cert\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.774545 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.774515 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:10:20.774545 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.774540 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:10:20.774701 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.774555 2563 projected.go:196] Error preparing data for projected volume kube-api-access-n95n7 for pod openshift-network-diagnostics/network-check-target-pzppf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:20.774803 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:20.774747 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7 podName:957fa4c9-20c3-4524-9b3d-4ed86dbda907 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:21.274619091 +0000 UTC m=+3.143229894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n95n7" (UniqueName: "kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7") pod "network-check-target-pzppf" (UID: "957fa4c9-20c3-4524-9b3d-4ed86dbda907") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:20.776649 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.776624 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtlhs\" (UniqueName: \"kubernetes.io/projected/8089c4bb-e3bb-4347-b575-4ebeae37a0bc-kube-api-access-xtlhs\") pod \"aws-ebs-csi-driver-node-k4bqq\" (UID: \"8089c4bb-e3bb-4347-b575-4ebeae37a0bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.776785 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.776766 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hqvw\" (UniqueName: \"kubernetes.io/projected/7bb73db5-1ffe-4b1e-9cbc-e6744f653d65-kube-api-access-9hqvw\") pod \"ovnkube-node-bvgbg\" (UID: \"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:20.777068 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.777051 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcc2f\" (UniqueName: \"kubernetes.io/projected/856900cf-180b-445f-8b7d-8616d1b93228-kube-api-access-mcc2f\") pod \"iptables-alerter-l9cjh\" (UID: \"856900cf-180b-445f-8b7d-8616d1b93228\") " pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.862084 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.861989 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-55vd4" Jul 12 22:10:20.870485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.870462 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vht46" Jul 12 22:10:20.881798 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.881771 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:20.886697 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.886670 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-chlj5" Jul 12 22:10:20.892853 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.892818 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7wm2n" Jul 12 22:10:20.900527 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.900502 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l9cjh" Jul 12 22:10:20.907470 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.907440 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qm5vb" Jul 12 22:10:20.916043 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.916018 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" Jul 12 22:10:20.921785 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:20.921766 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:21.172114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.172021 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:21.172265 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:21.172192 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:21.172323 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:21.172271 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs podName:71466df1-3f21-4c6a-acbe-758aeea5531d nodeName:}" failed. No retries permitted until 2026-07-12 22:10:22.172249873 +0000 UTC m=+4.040860674 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs") pod "network-metrics-daemon-gq8dd" (UID: "71466df1-3f21-4c6a-acbe-758aeea5531d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:21.273165 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.273132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:21.273313 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:21.273256 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:21.273313 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:21.273310 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret podName:87116643-5000-4846-a88b-a452ab0ef89e nodeName:}" failed. No retries permitted until 2026-07-12 22:10:22.273297356 +0000 UTC m=+4.141908158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret") pod "global-pull-secret-syncer-cd9nz" (UID: "87116643-5000-4846-a88b-a452ab0ef89e") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:21.373598 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.373557 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:21.373760 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:21.373721 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:10:21.373760 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:21.373745 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:10:21.373760 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:21.373760 2563 projected.go:196] Error preparing data for projected volume kube-api-access-n95n7 for pod openshift-network-diagnostics/network-check-target-pzppf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:21.373882 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:21.373829 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7 podName:957fa4c9-20c3-4524-9b3d-4ed86dbda907 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:22.373812495 +0000 UTC m=+4.242423299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n95n7" (UniqueName: "kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7") pod "network-check-target-pzppf" (UID: "957fa4c9-20c3-4524-9b3d-4ed86dbda907") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:21.591382 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.591327 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 22:05:20 +0000 UTC" deadline="2028-03-21 21:30:45.13103388 +0000 UTC" Jul 12 22:10:21.591698 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.591384 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14831h20m23.539672179s" Jul 12 22:10:21.594915 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:21.594759 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbd4733_347d_40b6_a124_7f8df70f0ec6.slice/crio-e96c6d4b2f315b9592787a160e7aa9993f27440e9768e97771c5cb456c756e4d WatchSource:0}: Error finding container e96c6d4b2f315b9592787a160e7aa9993f27440e9768e97771c5cb456c756e4d: Status 404 returned error can't find the container with id e96c6d4b2f315b9592787a160e7aa9993f27440e9768e97771c5cb456c756e4d Jul 12 22:10:21.614337 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:21.614311 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b7f93a_2be5_4673_8f1c_9a88435b4e91.slice/crio-b3e8165407d954e3fcbe3119ecf099cb7c150209daca10cec13fed961b721155 WatchSource:0}: Error finding container b3e8165407d954e3fcbe3119ecf099cb7c150209daca10cec13fed961b721155: Status 404 returned error can't find the container with id b3e8165407d954e3fcbe3119ecf099cb7c150209daca10cec13fed961b721155 Jul 12 22:10:21.615407 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:21.615324 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb73db5_1ffe_4b1e_9cbc_e6744f653d65.slice/crio-2dc7b3f428bc309002355974e2a5e95adf5fd93297695497a11221342242c67e WatchSource:0}: Error finding container 2dc7b3f428bc309002355974e2a5e95adf5fd93297695497a11221342242c67e: Status 404 returned error can't find the container with id 2dc7b3f428bc309002355974e2a5e95adf5fd93297695497a11221342242c67e Jul 12 22:10:21.615934 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:21.615837 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod255bf093_2970_45da_b373_4729e0e39673.slice/crio-ccc6b5d70826d5c7ffb222396fbf83eeb158cb082229b4bf43b6a35674e86562 WatchSource:0}: Error finding container ccc6b5d70826d5c7ffb222396fbf83eeb158cb082229b4bf43b6a35674e86562: Status 404 returned error can't find the container with id ccc6b5d70826d5c7ffb222396fbf83eeb158cb082229b4bf43b6a35674e86562 Jul 12 22:10:21.624808 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.624785 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:21.624913 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:21.624890 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:21.629307 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:21.629283 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb29b61f6_9dba_468c_adcd_146cbfc66a27.slice/crio-2872f245ec5088f9c7f0a9b6a316cab358ce4ff8272ebb3c05756d1431bcd39e WatchSource:0}: Error finding container 2872f245ec5088f9c7f0a9b6a316cab358ce4ff8272ebb3c05756d1431bcd39e: Status 404 returned error can't find the container with id 2872f245ec5088f9c7f0a9b6a316cab358ce4ff8272ebb3c05756d1431bcd39e Jul 12 22:10:21.629963 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:21.629944 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856900cf_180b_445f_8b7d_8616d1b93228.slice/crio-94420b63744077b6f89db8d86b9bb992ea251226a59faee1a5a164e5c8a9c5a2 WatchSource:0}: Error finding container 94420b63744077b6f89db8d86b9bb992ea251226a59faee1a5a164e5c8a9c5a2: Status 404 returned error can't find the container with id 94420b63744077b6f89db8d86b9bb992ea251226a59faee1a5a164e5c8a9c5a2 Jul 12 22:10:21.631957 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:21.631922 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e80707e_d52d_4b32_ac55_51afc01ccf59.slice/crio-023789ba4a19ab78babc10f2ea873e28eefb892af921f84603b2074b3e381567 WatchSource:0}: Error finding container 023789ba4a19ab78babc10f2ea873e28eefb892af921f84603b2074b3e381567: Status 404 returned error can't find the container with id 023789ba4a19ab78babc10f2ea873e28eefb892af921f84603b2074b3e381567 Jul 12 22:10:21.632835 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:21.632675 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8089c4bb_e3bb_4347_b575_4ebeae37a0bc.slice/crio-20d3f3760c5a8401738cca361511b96aad1b0c994c649dd074b811010787b724 WatchSource:0}: Error finding container 20d3f3760c5a8401738cca361511b96aad1b0c994c649dd074b811010787b724: Status 404 returned error can't find the container with id 20d3f3760c5a8401738cca361511b96aad1b0c994c649dd074b811010787b724 Jul 12 22:10:21.635033 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.635004 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l9cjh" event={"ID":"856900cf-180b-445f-8b7d-8616d1b93228","Type":"ContainerStarted","Data":"94420b63744077b6f89db8d86b9bb992ea251226a59faee1a5a164e5c8a9c5a2"} Jul 12 22:10:21.636430 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.636404 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qm5vb" event={"ID":"1ed266b7-f4d9-4768-83e8-493ee6015789","Type":"ContainerStarted","Data":"0a06e93736bd87d905f8d9018b6008cc56a959465437727a9ec469a0031304bf"} Jul 12 22:10:21.637475 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.637422 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7wm2n" event={"ID":"255bf093-2970-45da-b373-4729e0e39673","Type":"ContainerStarted","Data":"ccc6b5d70826d5c7ffb222396fbf83eeb158cb082229b4bf43b6a35674e86562"} Jul 12 22:10:21.639075 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.639051 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerStarted","Data":"2dc7b3f428bc309002355974e2a5e95adf5fd93297695497a11221342242c67e"} Jul 12 22:10:21.640435 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.640324 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chlj5" event={"ID":"b29b61f6-9dba-468c-adcd-146cbfc66a27","Type":"ContainerStarted","Data":"2872f245ec5088f9c7f0a9b6a316cab358ce4ff8272ebb3c05756d1431bcd39e"} Jul 12 22:10:21.641350 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.641331 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vht46" event={"ID":"84b7f93a-2be5-4673-8f1c-9a88435b4e91","Type":"ContainerStarted","Data":"b3e8165407d954e3fcbe3119ecf099cb7c150209daca10cec13fed961b721155"} Jul 12 22:10:21.642517 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:21.642498 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g7vtm" event={"ID":"1cbd4733-347d-40b6-a124-7f8df70f0ec6","Type":"ContainerStarted","Data":"e96c6d4b2f315b9592787a160e7aa9993f27440e9768e97771c5cb456c756e4d"} Jul 12 22:10:22.178976 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.178927 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:22.179162 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.179114 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:22.179221 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.179177 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs podName:71466df1-3f21-4c6a-acbe-758aeea5531d nodeName:}" failed. No retries permitted until 2026-07-12 22:10:24.179159764 +0000 UTC m=+6.047770582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs") pod "network-metrics-daemon-gq8dd" (UID: "71466df1-3f21-4c6a-acbe-758aeea5531d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:22.219526 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.219488 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 22:10:22.280459 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.280351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:22.280632 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.280554 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:22.280632 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.280622 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret podName:87116643-5000-4846-a88b-a452ab0ef89e nodeName:}" failed. No retries permitted until 2026-07-12 22:10:24.28060333 +0000 UTC m=+6.149214135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret") pod "global-pull-secret-syncer-cd9nz" (UID: "87116643-5000-4846-a88b-a452ab0ef89e") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:22.381218 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.381126 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:22.381409 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.381266 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:10:22.381409 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.381293 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:10:22.381409 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.381309 2563 projected.go:196] Error preparing data for projected volume kube-api-access-n95n7 for pod openshift-network-diagnostics/network-check-target-pzppf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:22.381409 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.381388 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7 podName:957fa4c9-20c3-4524-9b3d-4ed86dbda907 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:24.381352542 +0000 UTC m=+6.249963359 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n95n7" (UniqueName: "kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7") pod "network-check-target-pzppf" (UID: "957fa4c9-20c3-4524-9b3d-4ed86dbda907") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:22.592469 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.592399 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 22:05:20 +0000 UTC" deadline="2028-02-18 22:00:00.480774808 +0000 UTC" Jul 12 22:10:22.592469 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.592462 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14063h49m37.888316843s" Jul 12 22:10:22.625986 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.625961 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:22.626173 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.626090 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:22.626438 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.625961 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:22.626538 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:22.626512 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:22.671900 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.671803 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-55vd4" event={"ID":"4e80707e-d52d-4b32-ac55-51afc01ccf59","Type":"ContainerStarted","Data":"023789ba4a19ab78babc10f2ea873e28eefb892af921f84603b2074b3e381567"} Jul 12 22:10:22.677913 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.677878 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal" event={"ID":"73621ab645bf612f0f90ffad1d6de781","Type":"ContainerStarted","Data":"37a73e86249dcb96fd2d09fca7ffac7b1b03a1042e91b2a88a0d82dee0a159bb"} Jul 12 22:10:22.690123 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.690086 2563 generic.go:356] "Generic (PLEG): container finished" podID="864f4becf45250c71ccc80209b0b9c7f" containerID="059cde6a8f96b68a917750797c0d1d78804e509f5363b1b442ec45b5641ca119" exitCode=0 Jul 12 22:10:22.690280 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.690184 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" event={"ID":"864f4becf45250c71ccc80209b0b9c7f","Type":"ContainerDied","Data":"059cde6a8f96b68a917750797c0d1d78804e509f5363b1b442ec45b5641ca119"} Jul 12 22:10:22.714184 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.712998 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-44.ec2.internal" podStartSLOduration=3.712976376 podStartE2EDuration="3.712976376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:22.69730192 +0000 UTC m=+4.565912745" watchObservedRunningTime="2026-07-12 22:10:22.712976376 +0000 UTC m=+4.581587201" Jul 12 22:10:22.714184 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:22.714057 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" event={"ID":"8089c4bb-e3bb-4347-b575-4ebeae37a0bc","Type":"ContainerStarted","Data":"20d3f3760c5a8401738cca361511b96aad1b0c994c649dd074b811010787b724"} Jul 12 22:10:23.626314 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:23.625823 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:23.626314 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:23.625966 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:23.734076 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:23.734007 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" event={"ID":"864f4becf45250c71ccc80209b0b9c7f","Type":"ContainerStarted","Data":"a30a1eea349c2f63f846fd991ec1bdde547e247f99c01db2e9c52927a6c7c82d"} Jul 12 22:10:24.194495 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:24.194454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:24.194667 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.194637 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:24.194741 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.194707 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs podName:71466df1-3f21-4c6a-acbe-758aeea5531d nodeName:}" failed. No retries permitted until 2026-07-12 22:10:28.194687892 +0000 UTC m=+10.063298711 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs") pod "network-metrics-daemon-gq8dd" (UID: "71466df1-3f21-4c6a-acbe-758aeea5531d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:24.295104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:24.295064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:24.295286 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.295246 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:24.295351 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.295312 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret podName:87116643-5000-4846-a88b-a452ab0ef89e nodeName:}" failed. No retries permitted until 2026-07-12 22:10:28.295293351 +0000 UTC m=+10.163904156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret") pod "global-pull-secret-syncer-cd9nz" (UID: "87116643-5000-4846-a88b-a452ab0ef89e") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:24.396052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:24.396015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:24.396238 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.396207 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:10:24.396238 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.396231 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:10:24.396347 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.396243 2563 projected.go:196] Error preparing data for projected volume kube-api-access-n95n7 for pod openshift-network-diagnostics/network-check-target-pzppf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:24.396347 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.396302 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7 podName:957fa4c9-20c3-4524-9b3d-4ed86dbda907 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:28.396283946 +0000 UTC m=+10.264894752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n95n7" (UniqueName: "kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7") pod "network-check-target-pzppf" (UID: "957fa4c9-20c3-4524-9b3d-4ed86dbda907") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:24.626411 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:24.626314 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:24.626847 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.626466 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:24.626911 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:24.626858 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:24.627007 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:24.626951 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:25.625387 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:25.625343 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:25.625576 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:25.625512 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:26.625934 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:26.625900 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:26.626394 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:26.625917 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:26.626394 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:26.626032 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:26.626394 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:26.626112 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:27.625013 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:27.624936 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:27.625160 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:27.625026 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:28.227996 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:28.227953 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:28.228478 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.228108 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:28.228478 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.228170 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs podName:71466df1-3f21-4c6a-acbe-758aeea5531d nodeName:}" failed. No retries permitted until 2026-07-12 22:10:36.228151878 +0000 UTC m=+18.096762742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs") pod "network-metrics-daemon-gq8dd" (UID: "71466df1-3f21-4c6a-acbe-758aeea5531d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:28.329482 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:28.329306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:28.329482 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.329485 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:28.329711 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.329552 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret podName:87116643-5000-4846-a88b-a452ab0ef89e nodeName:}" failed. No retries permitted until 2026-07-12 22:10:36.329534027 +0000 UTC m=+18.198144828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret") pod "global-pull-secret-syncer-cd9nz" (UID: "87116643-5000-4846-a88b-a452ab0ef89e") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:28.430552 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:28.430509 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:28.430717 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.430698 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:10:28.430789 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.430723 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:10:28.430789 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.430738 2563 projected.go:196] Error preparing data for projected volume kube-api-access-n95n7 for pod openshift-network-diagnostics/network-check-target-pzppf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:28.430891 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.430798 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7 podName:957fa4c9-20c3-4524-9b3d-4ed86dbda907 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:36.430779132 +0000 UTC m=+18.299389954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n95n7" (UniqueName: "kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7") pod "network-check-target-pzppf" (UID: "957fa4c9-20c3-4524-9b3d-4ed86dbda907") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:28.625830 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:28.625753 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:28.626119 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:28.625784 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:28.626119 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.626107 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:28.626269 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:28.626208 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:29.625269 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:29.625234 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:29.625644 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:29.625403 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:30.625841 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:30.625141 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:30.625841 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:30.625272 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:30.625841 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:30.625697 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:30.625841 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:30.625791 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:31.624835 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:31.624805 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:31.625000 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:31.624917 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:32.628161 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:32.628079 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:32.628595 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:32.628205 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:32.628595 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:32.628585 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:32.628714 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:32.628678 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:33.625360 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:33.625326 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:33.625525 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:33.625488 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:34.625612 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:34.625572 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:34.626023 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:34.625624 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:34.626023 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:34.625722 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:34.626135 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:34.626046 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:35.625448 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:35.625405 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:35.625746 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:35.625551 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:36.292835 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:36.292802 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:36.293014 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.292943 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:36.293014 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.292996 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs podName:71466df1-3f21-4c6a-acbe-758aeea5531d nodeName:}" failed. No retries permitted until 2026-07-12 22:10:52.292981651 +0000 UTC m=+34.161592453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs") pod "network-metrics-daemon-gq8dd" (UID: "71466df1-3f21-4c6a-acbe-758aeea5531d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:36.393473 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:36.393437 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:36.393644 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.393538 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:36.393644 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.393600 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret podName:87116643-5000-4846-a88b-a452ab0ef89e nodeName:}" failed. No retries permitted until 2026-07-12 22:10:52.393582617 +0000 UTC m=+34.262193418 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret") pod "global-pull-secret-syncer-cd9nz" (UID: "87116643-5000-4846-a88b-a452ab0ef89e") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:36.494683 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:36.494649 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:36.494858 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.494844 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:10:36.494911 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.494866 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:10:36.494911 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.494881 2563 projected.go:196] Error preparing data for projected volume kube-api-access-n95n7 for pod openshift-network-diagnostics/network-check-target-pzppf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:36.494984 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.494935 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7 podName:957fa4c9-20c3-4524-9b3d-4ed86dbda907 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:52.49491912 +0000 UTC m=+34.363529927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n95n7" (UniqueName: "kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7") pod "network-check-target-pzppf" (UID: "957fa4c9-20c3-4524-9b3d-4ed86dbda907") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:36.625557 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:36.625479 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:36.625706 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:36.625485 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:36.625706 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.625612 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:36.625706 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:36.625684 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:37.625798 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:37.625760 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:37.626236 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:37.625906 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:38.629994 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:38.629958 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:38.630456 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:38.629958 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:38.630456 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:38.630176 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:38.630456 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:38.630058 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:39.624992 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:39.624953 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:39.625176 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:39.625085 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:40.630093 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.628870 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:40.630093 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:40.629171 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:40.630093 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.629566 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:40.630093 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:40.629651 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:40.767425 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.767356 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qm5vb" event={"ID":"1ed266b7-f4d9-4768-83e8-493ee6015789","Type":"ContainerStarted","Data":"651b75f076095007fb7a4a81f48aeac0bd7e1b231b9480589731147fa2090929"} Jul 12 22:10:40.769272 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.769248 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7wm2n" event={"ID":"255bf093-2970-45da-b373-4729e0e39673","Type":"ContainerStarted","Data":"15991def9ac90b1869869b9af19bde3eb2c953950b8aa8bfd25a74a9d8dceb65"} Jul 12 22:10:40.771421 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.771399 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:10:40.771804 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.771781 2563 generic.go:356] "Generic (PLEG): container finished" podID="7bb73db5-1ffe-4b1e-9cbc-e6744f653d65" containerID="6598924b566b8c6ec8c1c7821e180c8650619db89ac646e29e492988992c7c29" exitCode=1 Jul 12 22:10:40.771917 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.771857 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerDied","Data":"6598924b566b8c6ec8c1c7821e180c8650619db89ac646e29e492988992c7c29"} Jul 12 22:10:40.771977 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.771929 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerStarted","Data":"599adeb3dbd297ce17c87afeb91417ea6de0a32bd7fe5bc0960eb3f6b82eb943"} Jul 12 22:10:40.773497 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.773475 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" event={"ID":"8089c4bb-e3bb-4347-b575-4ebeae37a0bc","Type":"ContainerStarted","Data":"9a972c7972b4c3cc473b5de4fb24ad635c61b23063092e49c93e26b91e1cfced"} Jul 12 22:10:40.774976 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.774955 2563 generic.go:356] "Generic (PLEG): container finished" podID="b29b61f6-9dba-468c-adcd-146cbfc66a27" containerID="c63dc2931aadbc57412fc7ab7efc50d2f1e19fcc377599b319f42fcbc244378c" exitCode=0 Jul 12 22:10:40.775069 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.774996 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chlj5" event={"ID":"b29b61f6-9dba-468c-adcd-146cbfc66a27","Type":"ContainerDied","Data":"c63dc2931aadbc57412fc7ab7efc50d2f1e19fcc377599b319f42fcbc244378c"} Jul 12 22:10:40.776561 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.776520 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vht46" event={"ID":"84b7f93a-2be5-4673-8f1c-9a88435b4e91","Type":"ContainerStarted","Data":"906935a36c48b2be41d98685f39884142fa7be914660a8b17cc915323ff4b5a6"} Jul 12 22:10:40.778057 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.778031 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g7vtm" event={"ID":"1cbd4733-347d-40b6-a124-7f8df70f0ec6","Type":"ContainerStarted","Data":"c5662cda7e67afc54ed3dc02a865c9e6b6a6189681f79aa5aaf83db316bc5fc1"} Jul 12 22:10:40.779359 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.779338 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-55vd4" event={"ID":"4e80707e-d52d-4b32-ac55-51afc01ccf59","Type":"ContainerStarted","Data":"724f36f9e1df14fc65f0a21096f88bd5750d903af449cc518abda9031dc5487e"} Jul 12 22:10:40.784562 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.784526 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qm5vb" podStartSLOduration=3.8813989749999998 podStartE2EDuration="22.784514249s" totalImagesPullingTime="18.903115274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:40.784004878 +0000 UTC m=+22.652615704" watchObservedRunningTime="2026-07-12 22:10:40.784514249 +0000 UTC m=+22.653125105" Jul 12 22:10:40.784749 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.784725 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-44.ec2.internal" podStartSLOduration=21.784719599 podStartE2EDuration="21.784719599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:23.750141778 +0000 UTC m=+5.618752603" watchObservedRunningTime="2026-07-12 22:10:40.784719599 +0000 UTC m=+22.653330423" Jul 12 22:10:40.799608 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.799568 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vht46" podStartSLOduration=3.953923969 podStartE2EDuration="22.799554483s" totalImagesPullingTime="18.845630514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:40.799053494 +0000 UTC m=+22.667664314" watchObservedRunningTime="2026-07-12 22:10:40.799554483 +0000 UTC m=+22.668165307" Jul 12 22:10:40.815920 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.815866 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-55vd4" podStartSLOduration=3.9843082709999997 podStartE2EDuration="22.815846595s" totalImagesPullingTime="18.831538324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:40.815404728 +0000 UTC m=+22.684015551" watchObservedRunningTime="2026-07-12 22:10:40.815846595 +0000 UTC m=+22.684457420" Jul 12 22:10:40.864498 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:40.864454 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7wm2n" podStartSLOduration=4.018137808 podStartE2EDuration="22.864437994s" totalImagesPullingTime="18.846300186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:40.863618108 +0000 UTC m=+22.732228931" watchObservedRunningTime="2026-07-12 22:10:40.864437994 +0000 UTC m=+22.733048818" Jul 12 22:10:41.044047 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:41.043856 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:41.044527 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:41.044512 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:41.058413 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:41.058343 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g7vtm" podStartSLOduration=12.360464148 podStartE2EDuration="23.058329522s" totalImagesPullingTime="10.697865374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:40.882224382 +0000 UTC m=+22.750835206" watchObservedRunningTime="2026-07-12 22:10:41.058329522 +0000 UTC m=+22.926940345" Jul 12 22:10:41.625304 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:41.625268 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:41.625487 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:41.625392 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:41.785495 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:41.785464 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:10:41.786247 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:41.786019 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerStarted","Data":"8ce09f1395546feb0dbd7ee7af92b476878d3a2d29e5d786017f09707e360df5"} Jul 12 22:10:41.786247 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:41.786132 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerStarted","Data":"78aafe2d8e7488fe6a082ecaef31b0af424ac6c75849fcf0428e5d08c32f1e1b"} Jul 12 22:10:41.786247 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:41.786187 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerStarted","Data":"c7be35939e37340052acd36d9a21315ce49f71a14becd7a4050b434a34a16d0d"} Jul 12 22:10:41.786247 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:41.786231 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerStarted","Data":"5d622e2d7e220152e5beb68f0091d80f4a7eee42e15f3e24ab8a4e8cbb12c145"} Jul 12 22:10:42.280027 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.279986 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 22:10:42.472125 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.472043 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:42.472779 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.472760 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g7vtm" Jul 12 22:10:42.623591 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.623463 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T22:10:42.280006964Z","UUID":"c81d5b14-d537-4610-8a90-ad880bc1594c","Handler":null,"Name":"","Endpoint":""} Jul 12 22:10:42.624789 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.624764 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:42.624789 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.624789 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:42.624963 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:42.624882 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:42.625108 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:42.625018 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:42.625321 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.625301 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 22:10:42.625437 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.625337 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 22:10:42.790195 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.790107 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l9cjh" event={"ID":"856900cf-180b-445f-8b7d-8616d1b93228","Type":"ContainerStarted","Data":"507ceba368c313b75455a2560e0d804d34d7353a3837afd7b32e9b892e6b58d0"} Jul 12 22:10:42.792424 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.792392 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" event={"ID":"8089c4bb-e3bb-4347-b575-4ebeae37a0bc","Type":"ContainerStarted","Data":"3b53e8f72b5ba4ed183c097122a48aade9bc4d58230aea3a9fdf55203b7a627b"} Jul 12 22:10:42.805444 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:42.805386 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l9cjh" podStartSLOduration=5.973090388 podStartE2EDuration="24.805345261s" totalImagesPullingTime="18.832254873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:42.805074959 +0000 UTC m=+24.673685780" watchObservedRunningTime="2026-07-12 22:10:42.805345261 +0000 UTC m=+24.673956084" Jul 12 22:10:43.625416 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:43.625177 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:43.625581 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:43.625512 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:43.798271 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:43.798247 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:10:43.798739 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:43.798721 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerStarted","Data":"8183775369d3dcef55f8e2e807b577b756902150eb76834a4114dd5e2ffe2862"} Jul 12 22:10:44.629645 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:44.629615 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:44.629957 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:44.629614 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:44.629957 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:44.629743 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:44.629957 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:44.629794 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:44.804118 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:44.804083 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" event={"ID":"8089c4bb-e3bb-4347-b575-4ebeae37a0bc","Type":"ContainerStarted","Data":"803385d2c6087b53b0b561ad6fe0da5713062b51166412842482f391a845057f"} Jul 12 22:10:44.822913 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:44.822866 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k4bqq" podStartSLOduration=4.05001937 podStartE2EDuration="25.82284809s" totalImagesPullingTime="21.77282872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:44.822264116 +0000 UTC m=+26.690874939" watchObservedRunningTime="2026-07-12 22:10:44.82284809 +0000 UTC m=+26.691458914" Jul 12 22:10:45.625709 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:45.625673 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:45.625877 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:45.625802 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:46.625697 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.625483 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:46.626212 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.625551 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:46.626212 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:46.625727 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:46.626212 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:46.625805 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:46.811309 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.811282 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:10:46.811713 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.811687 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerStarted","Data":"9044995ec2fa72b263e4c2c5a181f75cdf2373c51f847f8c4cab760aef631b76"} Jul 12 22:10:46.812039 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.812012 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:46.812133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.812068 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:46.812133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.812101 2563 scope.go:117] "RemoveContainer" containerID="6598924b566b8c6ec8c1c7821e180c8650619db89ac646e29e492988992c7c29" Jul 12 22:10:46.813686 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.813662 2563 generic.go:356] "Generic (PLEG): container finished" podID="b29b61f6-9dba-468c-adcd-146cbfc66a27" containerID="fb0fea795313b37185742375604234a15589609fc83bebf226b9d033145e909f" exitCode=0 Jul 12 22:10:46.813780 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.813712 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chlj5" event={"ID":"b29b61f6-9dba-468c-adcd-146cbfc66a27","Type":"ContainerDied","Data":"fb0fea795313b37185742375604234a15589609fc83bebf226b9d033145e909f"} Jul 12 22:10:46.829456 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:46.829421 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:47.625911 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:47.625837 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:47.626320 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:47.625963 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:47.820442 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:47.820419 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:10:47.820865 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:47.820837 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" event={"ID":"7bb73db5-1ffe-4b1e-9cbc-e6744f653d65","Type":"ContainerStarted","Data":"275e4fc9c868b6493a6f6ee2e310de9766e1b9503f315a384016f53edc370a93"} Jul 12 22:10:47.821038 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:47.821018 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:47.836868 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:47.836848 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:10:47.863249 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:47.863203 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" podStartSLOduration=9.947790397 podStartE2EDuration="28.863187463s" totalImagesPullingTime="18.915397066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:47.861605115 +0000 UTC m=+29.730215938" watchObservedRunningTime="2026-07-12 22:10:47.863187463 +0000 UTC m=+29.731798285" Jul 12 22:10:48.038345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:48.038315 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gq8dd"] Jul 12 22:10:48.038482 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:48.038464 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:48.038604 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:48.038582 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:48.051344 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:48.051234 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cd9nz"] Jul 12 22:10:48.051344 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:48.051342 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:48.051504 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:48.051466 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:48.071081 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:48.071059 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pzppf"] Jul 12 22:10:48.071219 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:48.071167 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:48.071289 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:48.071266 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:48.824553 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:48.824520 2563 generic.go:356] "Generic (PLEG): container finished" podID="b29b61f6-9dba-468c-adcd-146cbfc66a27" containerID="69cf27729997aae0645d00182f4138747f41a99681eae2f7e0e7ff359f5e3430" exitCode=0 Jul 12 22:10:48.824967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:48.824617 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chlj5" event={"ID":"b29b61f6-9dba-468c-adcd-146cbfc66a27","Type":"ContainerDied","Data":"69cf27729997aae0645d00182f4138747f41a99681eae2f7e0e7ff359f5e3430"} Jul 12 22:10:49.625284 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:49.625131 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:49.625410 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:49.625134 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:49.625410 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:49.625359 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:49.625528 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:49.625488 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:49.625528 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:49.625134 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:49.625624 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:49.625607 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:50.830724 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:50.830677 2563 generic.go:356] "Generic (PLEG): container finished" podID="b29b61f6-9dba-468c-adcd-146cbfc66a27" containerID="b8cc862b343bdfd481976fc4ebac59515f338adc4226ab41df5df0c3930bb443" exitCode=0 Jul 12 22:10:50.831312 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:50.830740 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chlj5" event={"ID":"b29b61f6-9dba-468c-adcd-146cbfc66a27","Type":"ContainerDied","Data":"b8cc862b343bdfd481976fc4ebac59515f338adc4226ab41df5df0c3930bb443"} Jul 12 22:10:51.625214 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:51.625185 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:51.625408 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:51.625220 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:51.625408 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:51.625185 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:51.625408 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:51.625299 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gq8dd" podUID="71466df1-3f21-4c6a-acbe-758aeea5531d" Jul 12 22:10:51.625408 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:51.625355 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cd9nz" podUID="87116643-5000-4846-a88b-a452ab0ef89e" Jul 12 22:10:51.625587 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:51.625462 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pzppf" podUID="957fa4c9-20c3-4524-9b3d-4ed86dbda907" Jul 12 22:10:52.313503 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:52.313462 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:52.313907 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:52.313618 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:52.313907 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:52.313691 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs podName:71466df1-3f21-4c6a-acbe-758aeea5531d nodeName:}" failed. No retries permitted until 2026-07-12 22:11:24.31367032 +0000 UTC m=+66.182281121 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs") pod "network-metrics-daemon-gq8dd" (UID: "71466df1-3f21-4c6a-acbe-758aeea5531d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 22:10:52.414648 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:52.414615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:52.414809 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:52.414756 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:52.414847 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:52.414817 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret podName:87116643-5000-4846-a88b-a452ab0ef89e nodeName:}" failed. No retries permitted until 2026-07-12 22:11:24.414802596 +0000 UTC m=+66.283413397 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret") pod "global-pull-secret-syncer-cd9nz" (UID: "87116643-5000-4846-a88b-a452ab0ef89e") : object "kube-system"/"original-pull-secret" not registered Jul 12 22:10:52.515988 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:52.515952 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:52.516159 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:52.516113 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 22:10:52.516159 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:52.516135 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 22:10:52.516159 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:52.516147 2563 projected.go:196] Error preparing data for projected volume kube-api-access-n95n7 for pod openshift-network-diagnostics/network-check-target-pzppf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:52.516290 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:52.516199 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7 podName:957fa4c9-20c3-4524-9b3d-4ed86dbda907 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:24.51618504 +0000 UTC m=+66.384795842 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n95n7" (UniqueName: "kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7") pod "network-check-target-pzppf" (UID: "957fa4c9-20c3-4524-9b3d-4ed86dbda907") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 22:10:53.449335 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.449255 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-44.ec2.internal" event="NodeReady" Jul 12 22:10:53.449810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.449422 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 22:10:53.513418 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.513385 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-649755d64-dr2bt"] Jul 12 22:10:53.515223 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.515209 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.518707 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.518684 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 12 22:10:53.519076 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.519058 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 12 22:10:53.519312 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.519299 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hjmtz\"" Jul 12 22:10:53.519399 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.519344 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 12 22:10:53.536890 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.536871 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 12 22:10:53.542973 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.542951 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-649755d64-dr2bt"] Jul 12 22:10:53.606930 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.606901 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5bxkf"] Jul 12 22:10:53.608817 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.608779 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5bxkf" Jul 12 22:10:53.612547 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.612526 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 22:10:53.613055 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.613033 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 22:10:53.614426 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.614407 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 22:10:53.614532 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.614460 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nwvjz\"" Jul 12 22:10:53.622066 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.622047 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wlkph"] Jul 12 22:10:53.624119 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.624101 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.624975 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.624953 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:10:53.625076 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625037 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-image-registry-private-configuration\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.625141 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625088 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-installation-pull-secrets\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.625141 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625116 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-bound-sa-token\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.625241 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625153 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-trusted-ca\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.625241 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625176 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmgss\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-kube-api-access-lmgss\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.625241 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-ca-trust-extracted\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.625401 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625250 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.625401 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625274 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-certificates\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.625401 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625308 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:10:53.625543 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.625433 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:10:53.627863 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.627846 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 22:10:53.627952 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.627869 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 22:10:53.629242 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.629076 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 22:10:53.629664 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.629642 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 22:10:53.630184 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.630082 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 22:10:53.630525 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.630505 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5bxkf"] Jul 12 22:10:53.630602 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.630578 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-npkdh\"" Jul 12 22:10:53.630764 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.630633 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 22:10:53.630817 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.630787 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bgjrf\"" Jul 12 22:10:53.631492 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.631476 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9r9hf\"" Jul 12 22:10:53.651778 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.651755 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wlkph"] Jul 12 22:10:53.726577 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726445 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97597565-bdb0-469d-9f37-dc5018e77128-cert\") pod \"ingress-canary-5bxkf\" (UID: \"97597565-bdb0-469d-9f37-dc5018e77128\") " pod="openshift-ingress-canary/ingress-canary-5bxkf" Jul 12 22:10:53.726577 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.726577 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726567 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-certificates\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.726898 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726601 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c47a8b-861f-4216-98fc-ea06a6de52f7-tmp-dir\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.726898 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:53.726614 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 22:10:53.726898 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:53.726628 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649755d64-dr2bt: secret "image-registry-tls" not found Jul 12 22:10:53.726898 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726626 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-image-registry-private-configuration\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.726898 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:53.726731 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls podName:27ef56a7-6caf-4ed6-b5f4-dc446ac67697 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:54.226711168 +0000 UTC m=+36.095321987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls") pod "image-registry-649755d64-dr2bt" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697") : secret "image-registry-tls" not found Jul 12 22:10:53.726898 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726821 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-installation-pull-secrets\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.726898 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726854 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-bound-sa-token\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.727283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726891 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp8sq\" (UniqueName: \"kubernetes.io/projected/97597565-bdb0-469d-9f37-dc5018e77128-kube-api-access-pp8sq\") pod \"ingress-canary-5bxkf\" (UID: \"97597565-bdb0-469d-9f37-dc5018e77128\") " pod="openshift-ingress-canary/ingress-canary-5bxkf" Jul 12 22:10:53.727283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726927 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzf8m\" (UniqueName: \"kubernetes.io/projected/80c47a8b-861f-4216-98fc-ea06a6de52f7-kube-api-access-bzf8m\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.727283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726952 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80c47a8b-861f-4216-98fc-ea06a6de52f7-config-volume\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.727283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.726978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-trusted-ca\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.727283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.727003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmgss\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-kube-api-access-lmgss\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.727283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.727027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.727283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.727061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-ca-trust-extracted\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.727283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.727237 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-certificates\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.727741 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.727296 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-ca-trust-extracted\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.728502 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.728479 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-trusted-ca\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.731346 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.731322 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-image-registry-private-configuration\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.731346 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.731331 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-installation-pull-secrets\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.744110 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.744086 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmgss\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-kube-api-access-lmgss\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.744348 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.744330 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-bound-sa-token\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:53.828069 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.828034 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c47a8b-861f-4216-98fc-ea06a6de52f7-tmp-dir\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.828237 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.828099 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp8sq\" (UniqueName: \"kubernetes.io/projected/97597565-bdb0-469d-9f37-dc5018e77128-kube-api-access-pp8sq\") pod \"ingress-canary-5bxkf\" (UID: \"97597565-bdb0-469d-9f37-dc5018e77128\") " pod="openshift-ingress-canary/ingress-canary-5bxkf" Jul 12 22:10:53.828237 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.828118 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzf8m\" (UniqueName: \"kubernetes.io/projected/80c47a8b-861f-4216-98fc-ea06a6de52f7-kube-api-access-bzf8m\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.828237 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.828138 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80c47a8b-861f-4216-98fc-ea06a6de52f7-config-volume\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.828237 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.828153 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.828237 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.828178 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97597565-bdb0-469d-9f37-dc5018e77128-cert\") pod \"ingress-canary-5bxkf\" (UID: \"97597565-bdb0-469d-9f37-dc5018e77128\") " pod="openshift-ingress-canary/ingress-canary-5bxkf" Jul 12 22:10:53.828555 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:53.828285 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 22:10:53.828555 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:53.828352 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls podName:80c47a8b-861f-4216-98fc-ea06a6de52f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:54.328333413 +0000 UTC m=+36.196944235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls") pod "dns-default-wlkph" (UID: "80c47a8b-861f-4216-98fc-ea06a6de52f7") : secret "dns-default-metrics-tls" not found Jul 12 22:10:53.828555 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.828465 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c47a8b-861f-4216-98fc-ea06a6de52f7-tmp-dir\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.828824 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.828800 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80c47a8b-861f-4216-98fc-ea06a6de52f7-config-volume\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.830608 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.830591 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97597565-bdb0-469d-9f37-dc5018e77128-cert\") pod \"ingress-canary-5bxkf\" (UID: \"97597565-bdb0-469d-9f37-dc5018e77128\") " pod="openshift-ingress-canary/ingress-canary-5bxkf" Jul 12 22:10:53.846211 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.846177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp8sq\" (UniqueName: \"kubernetes.io/projected/97597565-bdb0-469d-9f37-dc5018e77128-kube-api-access-pp8sq\") pod \"ingress-canary-5bxkf\" (UID: \"97597565-bdb0-469d-9f37-dc5018e77128\") " pod="openshift-ingress-canary/ingress-canary-5bxkf" Jul 12 22:10:53.847573 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.847550 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzf8m\" (UniqueName: \"kubernetes.io/projected/80c47a8b-861f-4216-98fc-ea06a6de52f7-kube-api-access-bzf8m\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:53.918552 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:53.918514 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5bxkf" Jul 12 22:10:54.089975 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:54.089767 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5bxkf"] Jul 12 22:10:54.093593 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:10:54.093526 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97597565_bdb0_469d_9f37_dc5018e77128.slice/crio-5e38ea65df8d0f33f99898f2048b539e1fe85e7417775caddfac99ad07d2d6b4 WatchSource:0}: Error finding container 5e38ea65df8d0f33f99898f2048b539e1fe85e7417775caddfac99ad07d2d6b4: Status 404 returned error can't find the container with id 5e38ea65df8d0f33f99898f2048b539e1fe85e7417775caddfac99ad07d2d6b4 Jul 12 22:10:54.231916 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:54.231881 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:54.232087 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:54.232057 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 22:10:54.232153 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:54.232099 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649755d64-dr2bt: secret "image-registry-tls" not found Jul 12 22:10:54.232201 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:54.232191 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls podName:27ef56a7-6caf-4ed6-b5f4-dc446ac67697 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:55.232160001 +0000 UTC m=+37.100770829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls") pod "image-registry-649755d64-dr2bt" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697") : secret "image-registry-tls" not found Jul 12 22:10:54.332840 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:54.332746 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:54.332993 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:54.332881 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 22:10:54.332993 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:54.332956 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls podName:80c47a8b-861f-4216-98fc-ea06a6de52f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:55.332938631 +0000 UTC m=+37.201549435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls") pod "dns-default-wlkph" (UID: "80c47a8b-861f-4216-98fc-ea06a6de52f7") : secret "dns-default-metrics-tls" not found Jul 12 22:10:54.840671 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:54.840630 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5bxkf" event={"ID":"97597565-bdb0-469d-9f37-dc5018e77128","Type":"ContainerStarted","Data":"5e38ea65df8d0f33f99898f2048b539e1fe85e7417775caddfac99ad07d2d6b4"} Jul 12 22:10:55.239603 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:55.239509 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:55.239780 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:55.239660 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 22:10:55.239780 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:55.239703 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649755d64-dr2bt: secret "image-registry-tls" not found Jul 12 22:10:55.239780 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:55.239775 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls podName:27ef56a7-6caf-4ed6-b5f4-dc446ac67697 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:57.239753303 +0000 UTC m=+39.108364114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls") pod "image-registry-649755d64-dr2bt" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697") : secret "image-registry-tls" not found Jul 12 22:10:55.340331 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:55.340291 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:55.340552 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:55.340477 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 22:10:55.340618 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:55.340576 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls podName:80c47a8b-861f-4216-98fc-ea06a6de52f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:10:57.340553129 +0000 UTC m=+39.209163938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls") pod "dns-default-wlkph" (UID: "80c47a8b-861f-4216-98fc-ea06a6de52f7") : secret "dns-default-metrics-tls" not found Jul 12 22:10:55.846898 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:55.846869 2563 generic.go:356] "Generic (PLEG): container finished" podID="b29b61f6-9dba-468c-adcd-146cbfc66a27" containerID="e10c343c8bb81fe143bb3c84103403bbc822093a290b1dd633cb7671f18d1a67" exitCode=0 Jul 12 22:10:55.847259 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:55.846922 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chlj5" event={"ID":"b29b61f6-9dba-468c-adcd-146cbfc66a27","Type":"ContainerDied","Data":"e10c343c8bb81fe143bb3c84103403bbc822093a290b1dd633cb7671f18d1a67"} Jul 12 22:10:56.662884 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:56.662848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:10:56.770797 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:56.770762 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:10:56.853425 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:56.853391 2563 generic.go:356] "Generic (PLEG): container finished" podID="b29b61f6-9dba-468c-adcd-146cbfc66a27" containerID="0a49ac83acef22a70ac98c8e1219163a3c8ba509a4d384fa38e4a072ae3e0e31" exitCode=0 Jul 12 22:10:56.853840 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:56.853459 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chlj5" event={"ID":"b29b61f6-9dba-468c-adcd-146cbfc66a27","Type":"ContainerDied","Data":"0a49ac83acef22a70ac98c8e1219163a3c8ba509a4d384fa38e4a072ae3e0e31"} Jul 12 22:10:57.254285 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:57.254206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:10:57.254461 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:57.254330 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 22:10:57.254461 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:57.254343 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649755d64-dr2bt: secret "image-registry-tls" not found Jul 12 22:10:57.254461 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:57.254423 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls podName:27ef56a7-6caf-4ed6-b5f4-dc446ac67697 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:01.254405478 +0000 UTC m=+43.123016280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls") pod "image-registry-649755d64-dr2bt" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697") : secret "image-registry-tls" not found Jul 12 22:10:57.355114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:57.355075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:10:57.355279 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:57.355199 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 22:10:57.355279 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:10:57.355267 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls podName:80c47a8b-861f-4216-98fc-ea06a6de52f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:01.355249393 +0000 UTC m=+43.223860207 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls") pod "dns-default-wlkph" (UID: "80c47a8b-861f-4216-98fc-ea06a6de52f7") : secret "dns-default-metrics-tls" not found Jul 12 22:10:57.863803 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:57.863764 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chlj5" event={"ID":"b29b61f6-9dba-468c-adcd-146cbfc66a27","Type":"ContainerStarted","Data":"b521ed9a394371472be56c313d74beec7ff37f8d84c05c49ba359ab59dc042a3"} Jul 12 22:10:57.895576 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:57.895527 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-chlj5" podStartSLOduration=8.63443524 podStartE2EDuration="39.895512149s" totalImagesPullingTime="31.261076909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:57.894863388 +0000 UTC m=+39.763474223" watchObservedRunningTime="2026-07-12 22:10:57.895512149 +0000 UTC m=+39.764123008" Jul 12 22:10:58.092087 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:58.092046 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:10:58.164765 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:58.164677 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:10:58.618769 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:58.618735 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:10:58.690978 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:58.690949 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:10:58.867848 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:58.867818 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5bxkf" event={"ID":"97597565-bdb0-469d-9f37-dc5018e77128","Type":"ContainerStarted","Data":"3d367bf725bb3d35203cd31fbeeef3bd4c6dabcaed5f519572ff3cac60dae5f1"} Jul 12 22:10:58.912031 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:10:58.911977 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5bxkf" podStartSLOduration=2.181212185 podStartE2EDuration="5.911956132s" totalImagesPullingTime="3.730743947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:10:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:10:58.911387675 +0000 UTC m=+40.779998492" watchObservedRunningTime="2026-07-12 22:10:58.911956132 +0000 UTC m=+40.780566957" Jul 12 22:11:01.285497 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:01.285462 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:11:01.286014 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:01.285984 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 22:11:01.286075 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:01.286019 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649755d64-dr2bt: secret "image-registry-tls" not found Jul 12 22:11:01.286143 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:01.286093 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls podName:27ef56a7-6caf-4ed6-b5f4-dc446ac67697 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:09.286064343 +0000 UTC m=+51.154675162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls") pod "image-registry-649755d64-dr2bt" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697") : secret "image-registry-tls" not found Jul 12 22:11:01.387024 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:01.386982 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:11:01.387176 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:01.387100 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 22:11:01.387176 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:01.387149 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls podName:80c47a8b-861f-4216-98fc-ea06a6de52f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:09.387136451 +0000 UTC m=+51.255747253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls") pod "dns-default-wlkph" (UID: "80c47a8b-861f-4216-98fc-ea06a6de52f7") : secret "dns-default-metrics-tls" not found Jul 12 22:11:04.595112 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:04.595080 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:04.671828 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:04.671794 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:04.771499 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:04.771465 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:04.912319 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:04.912227 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:05.020337 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.020303 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:05.086278 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.086238 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:05.706018 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.705980 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:05.796062 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.796031 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:05.834674 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.834649 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-wtzwb"] Jul 12 22:11:05.878081 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.878041 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-wtzwb"] Jul 12 22:11:05.878257 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.878180 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:05.884334 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.884307 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 12 22:11:05.884555 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.884355 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 12 22:11:05.885152 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.885093 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 22:11:05.885243 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.885226 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-cmj5c\"" Jul 12 22:11:05.885782 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.885765 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.889071 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.889057 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 12 22:11:05.929081 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.929036 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-jx469"] Jul 12 22:11:05.949123 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.949090 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd"] Jul 12 22:11:05.949267 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.949175 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:05.953822 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.953798 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8b4kg\"" Jul 12 22:11:05.954666 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.954641 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 12 22:11:05.955141 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.954702 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.955235 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.954702 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 12 22:11:05.955235 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.954843 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 12 22:11:05.957659 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.957608 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 12 22:11:05.975143 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.975121 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd"] Jul 12 22:11:05.975143 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.975151 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-jx469"] Jul 12 22:11:05.975301 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.975242 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:05.977668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.977652 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 12 22:11:05.977966 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.977937 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 12 22:11:05.977966 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.977955 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 12 22:11:05.978114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.978021 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-dl9s7\"" Jul 12 22:11:05.978114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:05.978027 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 12 22:11:06.016808 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.016767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3164201-3898-4439-907f-3f9d20dd5c57-tmp\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.016808 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.016808 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3164201-3898-4439-907f-3f9d20dd5c57-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.017014 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.016831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3164201-3898-4439-907f-3f9d20dd5c57-serving-cert\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.017014 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.016892 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3164201-3898-4439-907f-3f9d20dd5c57-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.017014 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.016958 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d3164201-3898-4439-907f-3f9d20dd5c57-snapshots\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.017106 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.017017 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7jr4\" (UniqueName: \"kubernetes.io/projected/d3164201-3898-4439-907f-3f9d20dd5c57-kube-api-access-p7jr4\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.055263 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.055231 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:06.118154 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118117 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zm55\" (UniqueName: \"kubernetes.io/projected/cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf-kube-api-access-7zm55\") pod \"service-ca-operator-7695fc6c8c-s9vnd\" (UID: \"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.118154 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118155 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1dcff9c3-da5f-47d9-a082-7a79e9adad93-config\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.118415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7jr4\" (UniqueName: \"kubernetes.io/projected/d3164201-3898-4439-907f-3f9d20dd5c57-kube-api-access-p7jr4\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.118415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118209 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf-config\") pod \"service-ca-operator-7695fc6c8c-s9vnd\" (UID: \"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.118415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118225 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1dcff9c3-da5f-47d9-a082-7a79e9adad93-serving-cert\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.118415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118243 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf-serving-cert\") pod \"service-ca-operator-7695fc6c8c-s9vnd\" (UID: \"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.118415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118264 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3164201-3898-4439-907f-3f9d20dd5c57-tmp\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.118415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118284 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1dcff9c3-da5f-47d9-a082-7a79e9adad93-trusted-ca\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.118415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118316 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d3164201-3898-4439-907f-3f9d20dd5c57-snapshots\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.118415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118341 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3164201-3898-4439-907f-3f9d20dd5c57-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.118415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118390 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3164201-3898-4439-907f-3f9d20dd5c57-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.118828 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118427 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3164201-3898-4439-907f-3f9d20dd5c57-serving-cert\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.118828 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118451 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bg77\" (UniqueName: \"kubernetes.io/projected/1dcff9c3-da5f-47d9-a082-7a79e9adad93-kube-api-access-6bg77\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.118828 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118777 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3164201-3898-4439-907f-3f9d20dd5c57-tmp\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.118966 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118947 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3164201-3898-4439-907f-3f9d20dd5c57-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.119009 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.118989 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d3164201-3898-4439-907f-3f9d20dd5c57-snapshots\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.119118 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.119103 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3164201-3898-4439-907f-3f9d20dd5c57-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.122695 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.122677 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3164201-3898-4439-907f-3f9d20dd5c57-serving-cert\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.125865 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.125839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:06.127139 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.127107 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7jr4\" (UniqueName: \"kubernetes.io/projected/d3164201-3898-4439-907f-3f9d20dd5c57-kube-api-access-p7jr4\") pod \"insights-operator-68d9dd94cf-wtzwb\" (UID: \"d3164201-3898-4439-907f-3f9d20dd5c57\") " pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.187942 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.187902 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" Jul 12 22:11:06.202436 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.202404 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:06.219127 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.219104 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bg77\" (UniqueName: \"kubernetes.io/projected/1dcff9c3-da5f-47d9-a082-7a79e9adad93-kube-api-access-6bg77\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.219227 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.219165 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zm55\" (UniqueName: \"kubernetes.io/projected/cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf-kube-api-access-7zm55\") pod \"service-ca-operator-7695fc6c8c-s9vnd\" (UID: \"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.219227 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.219192 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1dcff9c3-da5f-47d9-a082-7a79e9adad93-config\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.219323 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.219225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf-config\") pod \"service-ca-operator-7695fc6c8c-s9vnd\" (UID: \"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.219323 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.219247 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1dcff9c3-da5f-47d9-a082-7a79e9adad93-serving-cert\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.219323 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.219275 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf-serving-cert\") pod \"service-ca-operator-7695fc6c8c-s9vnd\" (UID: \"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.219323 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.219304 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1dcff9c3-da5f-47d9-a082-7a79e9adad93-trusted-ca\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.220136 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.220105 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1dcff9c3-da5f-47d9-a082-7a79e9adad93-config\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.220344 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.220318 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf-config\") pod \"service-ca-operator-7695fc6c8c-s9vnd\" (UID: \"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.220666 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.220644 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1dcff9c3-da5f-47d9-a082-7a79e9adad93-trusted-ca\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.222174 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.222155 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1dcff9c3-da5f-47d9-a082-7a79e9adad93-serving-cert\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.222174 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.222158 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf-serving-cert\") pod \"service-ca-operator-7695fc6c8c-s9vnd\" (UID: \"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.235758 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.235730 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zm55\" (UniqueName: \"kubernetes.io/projected/cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf-kube-api-access-7zm55\") pod \"service-ca-operator-7695fc6c8c-s9vnd\" (UID: \"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.235876 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.235775 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bg77\" (UniqueName: \"kubernetes.io/projected/1dcff9c3-da5f-47d9-a082-7a79e9adad93-kube-api-access-6bg77\") pod \"console-operator-7866497494-jx469\" (UID: \"1dcff9c3-da5f-47d9-a082-7a79e9adad93\") " pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.258181 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.258147 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:06.283995 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.283965 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" Jul 12 22:11:06.341963 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.341884 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-wtzwb"] Jul 12 22:11:06.345164 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:06.345133 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3164201_3898_4439_907f_3f9d20dd5c57.slice/crio-b99a533782588b083a21a24dd6534bbdb1a214c494c3002dc7e4faae795c99a9 WatchSource:0}: Error finding container b99a533782588b083a21a24dd6534bbdb1a214c494c3002dc7e4faae795c99a9: Status 404 returned error can't find the container with id b99a533782588b083a21a24dd6534bbdb1a214c494c3002dc7e4faae795c99a9 Jul 12 22:11:06.425515 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.425485 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-jx469"] Jul 12 22:11:06.428472 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:06.428442 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dcff9c3_da5f_47d9_a082_7a79e9adad93.slice/crio-244f6fe0ab9e55be8ae7ebb029f1c7a2369e9897243e4e852ee23561b20fa9f9 WatchSource:0}: Error finding container 244f6fe0ab9e55be8ae7ebb029f1c7a2369e9897243e4e852ee23561b20fa9f9: Status 404 returned error can't find the container with id 244f6fe0ab9e55be8ae7ebb029f1c7a2369e9897243e4e852ee23561b20fa9f9 Jul 12 22:11:06.440443 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.440421 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd"] Jul 12 22:11:06.443521 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:06.443497 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb95c7eb_6ba2_49d9_a3e9_5b6eea580fcf.slice/crio-d271e08588ab3f1970f42cdb14db0df890a6d1c8cf30ac48c273d88c0bd5a72f WatchSource:0}: Error finding container d271e08588ab3f1970f42cdb14db0df890a6d1c8cf30ac48c273d88c0bd5a72f: Status 404 returned error can't find the container with id d271e08588ab3f1970f42cdb14db0df890a6d1c8cf30ac48c273d88c0bd5a72f Jul 12 22:11:06.887089 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.887054 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" event={"ID":"d3164201-3898-4439-907f-3f9d20dd5c57","Type":"ContainerStarted","Data":"b99a533782588b083a21a24dd6534bbdb1a214c494c3002dc7e4faae795c99a9"} Jul 12 22:11:06.888164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.888142 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" event={"ID":"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf","Type":"ContainerStarted","Data":"d271e08588ab3f1970f42cdb14db0df890a6d1c8cf30ac48c273d88c0bd5a72f"} Jul 12 22:11:06.889079 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:06.889060 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jx469" event={"ID":"1dcff9c3-da5f-47d9-a082-7a79e9adad93","Type":"ContainerStarted","Data":"244f6fe0ab9e55be8ae7ebb029f1c7a2369e9897243e4e852ee23561b20fa9f9"} Jul 12 22:11:08.740924 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:08.740880 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:08.841129 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:08.841089 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:08.913184 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:08.913099 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:09.347118 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:09.347075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:11:09.347298 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:09.347261 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 22:11:09.347298 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:09.347281 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649755d64-dr2bt: secret "image-registry-tls" not found Jul 12 22:11:09.347476 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:09.347338 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls podName:27ef56a7-6caf-4ed6-b5f4-dc446ac67697 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:25.347322501 +0000 UTC m=+67.215933304 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls") pod "image-registry-649755d64-dr2bt" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697") : secret "image-registry-tls" not found Jul 12 22:11:09.448259 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:09.448220 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:11:09.448473 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:09.448378 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 22:11:09.448473 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:09.448447 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls podName:80c47a8b-861f-4216-98fc-ea06a6de52f7 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:25.448432007 +0000 UTC m=+67.317042809 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls") pod "dns-default-wlkph" (UID: "80c47a8b-861f-4216-98fc-ea06a6de52f7") : secret "dns-default-metrics-tls" not found Jul 12 22:11:10.520354 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.520317 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g"] Jul 12 22:11:10.533238 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.533213 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g"] Jul 12 22:11:10.533435 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.533325 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" Jul 12 22:11:10.537148 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.536968 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 12 22:11:10.537148 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.536992 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 12 22:11:10.537148 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.537001 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 12 22:11:10.537148 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.537040 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xtcpb\"" Jul 12 22:11:10.537148 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.536968 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 12 22:11:10.539203 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.539183 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86"] Jul 12 22:11:10.555219 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.555196 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86"] Jul 12 22:11:10.555330 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.555282 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.561866 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.559100 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 12 22:11:10.657836 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.657796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b3cbd7f-f581-4cf2-ab6b-57652efe60c6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6584f9b58-d7f8g\" (UID: \"6b3cbd7f-f581-4cf2-ab6b-57652efe60c6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" Jul 12 22:11:10.658028 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.657871 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdgnh\" (UniqueName: \"kubernetes.io/projected/04791549-c8ab-4d27-b18e-e0db220c16b0-kube-api-access-xdgnh\") pod \"klusterlet-addon-workmgr-f678588b-c7q86\" (UID: \"04791549-c8ab-4d27-b18e-e0db220c16b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.658028 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.657902 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/04791549-c8ab-4d27-b18e-e0db220c16b0-klusterlet-config\") pod \"klusterlet-addon-workmgr-f678588b-c7q86\" (UID: \"04791549-c8ab-4d27-b18e-e0db220c16b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.658028 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.657939 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrjl2\" (UniqueName: \"kubernetes.io/projected/6b3cbd7f-f581-4cf2-ab6b-57652efe60c6-kube-api-access-rrjl2\") pod \"managed-serviceaccount-addon-agent-6584f9b58-d7f8g\" (UID: \"6b3cbd7f-f581-4cf2-ab6b-57652efe60c6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" Jul 12 22:11:10.658147 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.658018 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04791549-c8ab-4d27-b18e-e0db220c16b0-tmp\") pod \"klusterlet-addon-workmgr-f678588b-c7q86\" (UID: \"04791549-c8ab-4d27-b18e-e0db220c16b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.758608 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.758570 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b3cbd7f-f581-4cf2-ab6b-57652efe60c6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6584f9b58-d7f8g\" (UID: \"6b3cbd7f-f581-4cf2-ab6b-57652efe60c6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" Jul 12 22:11:10.758868 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.758619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdgnh\" (UniqueName: \"kubernetes.io/projected/04791549-c8ab-4d27-b18e-e0db220c16b0-kube-api-access-xdgnh\") pod \"klusterlet-addon-workmgr-f678588b-c7q86\" (UID: \"04791549-c8ab-4d27-b18e-e0db220c16b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.758868 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.758635 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/04791549-c8ab-4d27-b18e-e0db220c16b0-klusterlet-config\") pod \"klusterlet-addon-workmgr-f678588b-c7q86\" (UID: \"04791549-c8ab-4d27-b18e-e0db220c16b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.758868 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.758660 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrjl2\" (UniqueName: \"kubernetes.io/projected/6b3cbd7f-f581-4cf2-ab6b-57652efe60c6-kube-api-access-rrjl2\") pod \"managed-serviceaccount-addon-agent-6584f9b58-d7f8g\" (UID: \"6b3cbd7f-f581-4cf2-ab6b-57652efe60c6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" Jul 12 22:11:10.758868 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.758826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04791549-c8ab-4d27-b18e-e0db220c16b0-tmp\") pod \"klusterlet-addon-workmgr-f678588b-c7q86\" (UID: \"04791549-c8ab-4d27-b18e-e0db220c16b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.759255 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.759222 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04791549-c8ab-4d27-b18e-e0db220c16b0-tmp\") pod \"klusterlet-addon-workmgr-f678588b-c7q86\" (UID: \"04791549-c8ab-4d27-b18e-e0db220c16b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.761344 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.761321 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/04791549-c8ab-4d27-b18e-e0db220c16b0-klusterlet-config\") pod \"klusterlet-addon-workmgr-f678588b-c7q86\" (UID: \"04791549-c8ab-4d27-b18e-e0db220c16b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.761792 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.761769 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b3cbd7f-f581-4cf2-ab6b-57652efe60c6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6584f9b58-d7f8g\" (UID: \"6b3cbd7f-f581-4cf2-ab6b-57652efe60c6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" Jul 12 22:11:10.768746 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.768717 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrjl2\" (UniqueName: \"kubernetes.io/projected/6b3cbd7f-f581-4cf2-ab6b-57652efe60c6-kube-api-access-rrjl2\") pod \"managed-serviceaccount-addon-agent-6584f9b58-d7f8g\" (UID: \"6b3cbd7f-f581-4cf2-ab6b-57652efe60c6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" Jul 12 22:11:10.769354 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.769332 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdgnh\" (UniqueName: \"kubernetes.io/projected/04791549-c8ab-4d27-b18e-e0db220c16b0-kube-api-access-xdgnh\") pod \"klusterlet-addon-workmgr-f678588b-c7q86\" (UID: \"04791549-c8ab-4d27-b18e-e0db220c16b0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.852307 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.852273 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" Jul 12 22:11:10.865569 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.865532 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:10.906513 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.906480 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" event={"ID":"d3164201-3898-4439-907f-3f9d20dd5c57","Type":"ContainerStarted","Data":"f4ab967e5866d64dd6eb496e9510a70585acc9dee6320bdb6b7419d4e67f32eb"} Jul 12 22:11:10.908666 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.908624 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" event={"ID":"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf","Type":"ContainerStarted","Data":"22e38599ab3123a1105be8872c786610000e39ce37f224e7beded896319d8acf"} Jul 12 22:11:10.911795 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.911027 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/0.log" Jul 12 22:11:10.911795 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.911065 2563 generic.go:356] "Generic (PLEG): container finished" podID="1dcff9c3-da5f-47d9-a082-7a79e9adad93" containerID="fd49efaf56665e4c2eda63db647fdeb7f6f57ce4746f7e5f0ba21253d0bc9ea5" exitCode=255 Jul 12 22:11:10.911795 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.911148 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jx469" event={"ID":"1dcff9c3-da5f-47d9-a082-7a79e9adad93","Type":"ContainerDied","Data":"fd49efaf56665e4c2eda63db647fdeb7f6f57ce4746f7e5f0ba21253d0bc9ea5"} Jul 12 22:11:10.911795 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.911458 2563 scope.go:117] "RemoveContainer" containerID="fd49efaf56665e4c2eda63db647fdeb7f6f57ce4746f7e5f0ba21253d0bc9ea5" Jul 12 22:11:10.930562 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.930440 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" podStartSLOduration=2.032735825 podStartE2EDuration="5.930421408s" totalImagesPullingTime="3.897685583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:10.928917442 +0000 UTC m=+52.797528267" watchObservedRunningTime="2026-07-12 22:11:10.930421408 +0000 UTC m=+52.799032236" Jul 12 22:11:10.978696 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:10.977722 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" podStartSLOduration=2.1775403349999998 podStartE2EDuration="5.977705946s" totalImagesPullingTime="3.800165611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:10.977573461 +0000 UTC m=+52.846184286" watchObservedRunningTime="2026-07-12 22:11:10.977705946 +0000 UTC m=+52.846316761" Jul 12 22:11:11.010356 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.008662 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zcx4l"] Jul 12 22:11:11.037427 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.036759 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.043704 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.037963 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zcx4l"] Jul 12 22:11:11.043704 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.039912 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5sgr4\"" Jul 12 22:11:11.043704 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.040137 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 22:11:11.043704 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.040348 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 22:11:11.076026 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.075992 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:11.081324 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.081270 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g"] Jul 12 22:11:11.086835 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:11.086806 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b3cbd7f_f581_4cf2_ab6b_57652efe60c6.slice/crio-2f030b0db04beb18e5d25760ba7e4f5db744c02492cebed867fbcb6ae1ce4766 WatchSource:0}: Error finding container 2f030b0db04beb18e5d25760ba7e4f5db744c02492cebed867fbcb6ae1ce4766: Status 404 returned error can't find the container with id 2f030b0db04beb18e5d25760ba7e4f5db744c02492cebed867fbcb6ae1ce4766 Jul 12 22:11:11.105824 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.105741 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86"] Jul 12 22:11:11.109921 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:11.109881 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04791549_c8ab_4d27_b18e_e0db220c16b0.slice/crio-c59b59b35ba2cf383d3dafab6238a0b4b193a96287b474a14429d6d32fd08053 WatchSource:0}: Error finding container c59b59b35ba2cf383d3dafab6238a0b4b193a96287b474a14429d6d32fd08053: Status 404 returned error can't find the container with id c59b59b35ba2cf383d3dafab6238a0b4b193a96287b474a14429d6d32fd08053 Jul 12 22:11:11.163913 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.163883 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djkw5\" (UniqueName: \"kubernetes.io/projected/9842646b-932e-4847-8502-d2ecbad5e029-kube-api-access-djkw5\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.164072 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.163926 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.164072 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.164033 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9842646b-932e-4847-8502-d2ecbad5e029-data-volume\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.164159 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.164081 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9842646b-932e-4847-8502-d2ecbad5e029-crio-socket\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.164159 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.164131 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.176948 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.176914 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:11.264760 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.264717 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.264956 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.264774 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djkw5\" (UniqueName: \"kubernetes.io/projected/9842646b-932e-4847-8502-d2ecbad5e029-kube-api-access-djkw5\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.264956 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.264800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.264956 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.264842 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9842646b-932e-4847-8502-d2ecbad5e029-data-volume\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.264956 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.264877 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9842646b-932e-4847-8502-d2ecbad5e029-crio-socket\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.264956 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:11.264884 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 22:11:11.264956 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:11.264935 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 22:11:11.265246 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:11.264961 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:11.764939802 +0000 UTC m=+53.633550633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : secret "insights-runtime-extractor-tls" not found Jul 12 22:11:11.265246 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:11.264984 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:11.764973806 +0000 UTC m=+53.633584613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : configmap "kube-rbac-proxy" not found Jul 12 22:11:11.265246 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.265056 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9842646b-932e-4847-8502-d2ecbad5e029-crio-socket\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.265246 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.265241 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9842646b-932e-4847-8502-d2ecbad5e029-data-volume\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.277194 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.277162 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djkw5\" (UniqueName: \"kubernetes.io/projected/9842646b-932e-4847-8502-d2ecbad5e029-kube-api-access-djkw5\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.386772 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.386671 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:11.456899 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.456863 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:11.733996 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.733913 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:11.768021 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.767982 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.768210 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.768115 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:11.768210 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:11.768133 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 22:11:11.768210 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:11.768209 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:12.768184773 +0000 UTC m=+54.636795577 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : configmap "kube-rbac-proxy" not found Jul 12 22:11:11.768390 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:11.768227 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 22:11:11.768390 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:11.768288 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:12.768271369 +0000 UTC m=+54.636882186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : secret "insights-runtime-extractor-tls" not found Jul 12 22:11:11.833691 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.833652 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:11.907812 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.907775 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:11.917111 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.917071 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" event={"ID":"04791549-c8ab-4d27-b18e-e0db220c16b0","Type":"ContainerStarted","Data":"c59b59b35ba2cf383d3dafab6238a0b4b193a96287b474a14429d6d32fd08053"} Jul 12 22:11:11.919098 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.919079 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:11:11.919779 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.919492 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/0.log" Jul 12 22:11:11.919779 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.919527 2563 generic.go:356] "Generic (PLEG): container finished" podID="1dcff9c3-da5f-47d9-a082-7a79e9adad93" containerID="cefeebf7e795275378c0e13ed9d2585258f278d4bbdd62330accab3cc30a4841" exitCode=255 Jul 12 22:11:11.919957 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.919802 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jx469" event={"ID":"1dcff9c3-da5f-47d9-a082-7a79e9adad93","Type":"ContainerDied","Data":"cefeebf7e795275378c0e13ed9d2585258f278d4bbdd62330accab3cc30a4841"} Jul 12 22:11:11.919957 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.919856 2563 scope.go:117] "RemoveContainer" containerID="fd49efaf56665e4c2eda63db647fdeb7f6f57ce4746f7e5f0ba21253d0bc9ea5" Jul 12 22:11:11.919957 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.919904 2563 scope.go:117] "RemoveContainer" containerID="cefeebf7e795275378c0e13ed9d2585258f278d4bbdd62330accab3cc30a4841" Jul 12 22:11:11.920338 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:11.920114 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jx469_openshift-console-operator(1dcff9c3-da5f-47d9-a082-7a79e9adad93)\"" pod="openshift-console-operator/console-operator-7866497494-jx469" podUID="1dcff9c3-da5f-47d9-a082-7a79e9adad93" Jul 12 22:11:11.921881 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:11.921857 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" event={"ID":"6b3cbd7f-f581-4cf2-ab6b-57652efe60c6","Type":"ContainerStarted","Data":"2f030b0db04beb18e5d25760ba7e4f5db744c02492cebed867fbcb6ae1ce4766"} Jul 12 22:11:12.165361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:12.165318 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:12.286958 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:12.286922 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:12.777102 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:12.776919 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:12.777102 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:12.776993 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:12.777102 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:12.777095 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 22:11:12.777707 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:12.777116 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 22:11:12.777707 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:12.777181 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:14.777159169 +0000 UTC m=+56.645769987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : configmap "kube-rbac-proxy" not found Jul 12 22:11:12.777707 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:12.777212 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:14.777193327 +0000 UTC m=+56.645804147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : secret "insights-runtime-extractor-tls" not found Jul 12 22:11:12.828580 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:12.828351 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vht46_84b7f93a-2be5-4673-8f1c-9a88435b4e91/dns-node-resolver/0.log" Jul 12 22:11:12.927684 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:12.927655 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:11:12.928154 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:12.928052 2563 scope.go:117] "RemoveContainer" containerID="cefeebf7e795275378c0e13ed9d2585258f278d4bbdd62330accab3cc30a4841" Jul 12 22:11:12.928266 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:12.928247 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jx469_openshift-console-operator(1dcff9c3-da5f-47d9-a082-7a79e9adad93)\"" pod="openshift-console-operator/console-operator-7866497494-jx469" podUID="1dcff9c3-da5f-47d9-a082-7a79e9adad93" Jul 12 22:11:13.227538 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.227507 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7wm2n_255bf093-2970-45da-b373-4729e0e39673/node-ca/0.log" Jul 12 22:11:13.304471 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.304429 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:13.320384 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.320326 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr"] Jul 12 22:11:13.325500 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.325104 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr" Jul 12 22:11:13.328243 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.328217 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4zw2p\"" Jul 12 22:11:13.328469 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.328444 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 12 22:11:13.328573 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.328494 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 12 22:11:13.337058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.335791 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr"] Jul 12 22:11:13.400438 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.400397 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:13.483537 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.483456 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x28g8\" (UniqueName: \"kubernetes.io/projected/47b707c0-46ba-4dab-8d0c-311ffc7d7e6e-kube-api-access-x28g8\") pod \"migrator-7f59b54f6c-nblhr\" (UID: \"47b707c0-46ba-4dab-8d0c-311ffc7d7e6e\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr" Jul 12 22:11:13.531717 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.531609 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:13.584792 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.584702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x28g8\" (UniqueName: \"kubernetes.io/projected/47b707c0-46ba-4dab-8d0c-311ffc7d7e6e-kube-api-access-x28g8\") pod \"migrator-7f59b54f6c-nblhr\" (UID: \"47b707c0-46ba-4dab-8d0c-311ffc7d7e6e\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr" Jul 12 22:11:13.593722 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.593689 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x28g8\" (UniqueName: \"kubernetes.io/projected/47b707c0-46ba-4dab-8d0c-311ffc7d7e6e-kube-api-access-x28g8\") pod \"migrator-7f59b54f6c-nblhr\" (UID: \"47b707c0-46ba-4dab-8d0c-311ffc7d7e6e\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr" Jul 12 22:11:13.637864 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.637821 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr" Jul 12 22:11:13.696539 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.696504 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-74xxw"] Jul 12 22:11:13.700393 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.700350 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:13.703142 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.703110 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 12 22:11:13.703142 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.703134 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xg58z\"" Jul 12 22:11:13.703336 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.703153 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 12 22:11:13.704110 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.703872 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 12 22:11:13.704110 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.703872 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 12 22:11:13.708984 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.708958 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-74xxw"] Jul 12 22:11:13.750744 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.750654 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:13.786793 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.786762 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7n5m\" (UniqueName: \"kubernetes.io/projected/b8fdbbdd-90cd-468f-a897-394f3ef085d2-kube-api-access-m7n5m\") pod \"service-ca-79f68b9f4f-74xxw\" (UID: \"b8fdbbdd-90cd-468f-a897-394f3ef085d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:13.787257 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.786833 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8fdbbdd-90cd-468f-a897-394f3ef085d2-signing-key\") pod \"service-ca-79f68b9f4f-74xxw\" (UID: \"b8fdbbdd-90cd-468f-a897-394f3ef085d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:13.787257 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.786888 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8fdbbdd-90cd-468f-a897-394f3ef085d2-signing-cabundle\") pod \"service-ca-79f68b9f4f-74xxw\" (UID: \"b8fdbbdd-90cd-468f-a897-394f3ef085d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:13.849902 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.849859 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:13.887559 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.887515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7n5m\" (UniqueName: \"kubernetes.io/projected/b8fdbbdd-90cd-468f-a897-394f3ef085d2-kube-api-access-m7n5m\") pod \"service-ca-79f68b9f4f-74xxw\" (UID: \"b8fdbbdd-90cd-468f-a897-394f3ef085d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:13.887740 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.887600 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8fdbbdd-90cd-468f-a897-394f3ef085d2-signing-key\") pod \"service-ca-79f68b9f4f-74xxw\" (UID: \"b8fdbbdd-90cd-468f-a897-394f3ef085d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:13.887740 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.887671 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8fdbbdd-90cd-468f-a897-394f3ef085d2-signing-cabundle\") pod \"service-ca-79f68b9f4f-74xxw\" (UID: \"b8fdbbdd-90cd-468f-a897-394f3ef085d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:13.888506 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.888468 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8fdbbdd-90cd-468f-a897-394f3ef085d2-signing-cabundle\") pod \"service-ca-79f68b9f4f-74xxw\" (UID: \"b8fdbbdd-90cd-468f-a897-394f3ef085d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:13.891216 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.891193 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8fdbbdd-90cd-468f-a897-394f3ef085d2-signing-key\") pod \"service-ca-79f68b9f4f-74xxw\" (UID: \"b8fdbbdd-90cd-468f-a897-394f3ef085d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:13.897552 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:13.897527 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7n5m\" (UniqueName: \"kubernetes.io/projected/b8fdbbdd-90cd-468f-a897-394f3ef085d2-kube-api-access-m7n5m\") pod \"service-ca-79f68b9f4f-74xxw\" (UID: \"b8fdbbdd-90cd-468f-a897-394f3ef085d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:14.013742 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:14.013654 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" Jul 12 22:11:14.027439 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:14.027395 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5bxkf_97597565-bdb0-469d-9f37-dc5018e77128/serve-healthcheck-canary/0.log" Jul 12 22:11:14.622911 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:14.622873 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:14.796156 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:14.796114 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:14.796712 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:14.796239 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:14.796712 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:14.796244 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 22:11:14.796712 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:14.796402 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 22:11:14.796712 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:14.796453 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:18.79643485 +0000 UTC m=+60.665045651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : secret "insights-runtime-extractor-tls" not found Jul 12 22:11:14.796712 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:14.796499 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:18.796483082 +0000 UTC m=+60.665093899 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : configmap "kube-rbac-proxy" not found Jul 12 22:11:15.509520 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.509283 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr"] Jul 12 22:11:15.512796 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:15.512767 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b707c0_46ba_4dab_8d0c_311ffc7d7e6e.slice/crio-a67f4e0e69114500663ac1cb719f8cc96fa07de7151b7bc50665cd3be1a25966 WatchSource:0}: Error finding container a67f4e0e69114500663ac1cb719f8cc96fa07de7151b7bc50665cd3be1a25966: Status 404 returned error can't find the container with id a67f4e0e69114500663ac1cb719f8cc96fa07de7151b7bc50665cd3be1a25966 Jul 12 22:11:15.521706 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.521683 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-74xxw"] Jul 12 22:11:15.525119 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:15.525060 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8fdbbdd_90cd_468f_a897_394f3ef085d2.slice/crio-5ccca2f626ae81bff6299546f045e55e4c9ea2502ba07426bb2e8cccf6e4afab WatchSource:0}: Error finding container 5ccca2f626ae81bff6299546f045e55e4c9ea2502ba07426bb2e8cccf6e4afab: Status 404 returned error can't find the container with id 5ccca2f626ae81bff6299546f045e55e4c9ea2502ba07426bb2e8cccf6e4afab Jul 12 22:11:15.939015 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.938974 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" event={"ID":"6b3cbd7f-f581-4cf2-ab6b-57652efe60c6","Type":"ContainerStarted","Data":"4eca0f5f026e00a158780f9e5efd89dae26a75b027665026216b01fd2d74d947"} Jul 12 22:11:15.940469 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.940443 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" event={"ID":"b8fdbbdd-90cd-468f-a897-394f3ef085d2","Type":"ContainerStarted","Data":"7ea3e9985b489b1dad7d3441f0aa8b61c3161636f908d703de9ea80828b4c33f"} Jul 12 22:11:15.940615 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.940488 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" event={"ID":"b8fdbbdd-90cd-468f-a897-394f3ef085d2","Type":"ContainerStarted","Data":"5ccca2f626ae81bff6299546f045e55e4c9ea2502ba07426bb2e8cccf6e4afab"} Jul 12 22:11:15.942008 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.941984 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" event={"ID":"04791549-c8ab-4d27-b18e-e0db220c16b0","Type":"ContainerStarted","Data":"f6958bf58a38ec3f7ed9ede45bd9e6c7826f83310c5dca0be75325a766a02aa8"} Jul 12 22:11:15.942277 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.942127 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:15.943272 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.943244 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr" event={"ID":"47b707c0-46ba-4dab-8d0c-311ffc7d7e6e","Type":"ContainerStarted","Data":"a67f4e0e69114500663ac1cb719f8cc96fa07de7151b7bc50665cd3be1a25966"} Jul 12 22:11:15.943802 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.943787 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" Jul 12 22:11:15.957079 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.957042 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584f9b58-d7f8g" podStartSLOduration=1.685878999 podStartE2EDuration="5.957030276s" totalImagesPullingTime="4.271151277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:15.95676605 +0000 UTC m=+57.825376876" watchObservedRunningTime="2026-07-12 22:11:15.957030276 +0000 UTC m=+57.825641099" Jul 12 22:11:15.974753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:15.974708 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-74xxw" podStartSLOduration=2.974691949 podStartE2EDuration="2.974691949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:15.973478641 +0000 UTC m=+57.842089465" watchObservedRunningTime="2026-07-12 22:11:15.974691949 +0000 UTC m=+57.843302774" Jul 12 22:11:16.000294 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:16.000246 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f678588b-c7q86" podStartSLOduration=1.735896704 podStartE2EDuration="6.000236637s" totalImagesPullingTime="4.264339933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:15.999429022 +0000 UTC m=+57.868039847" watchObservedRunningTime="2026-07-12 22:11:16.000236637 +0000 UTC m=+57.868847505" Jul 12 22:11:16.258866 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:16.258785 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:16.258866 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:16.258863 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:16.259296 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:16.259281 2563 scope.go:117] "RemoveContainer" containerID="cefeebf7e795275378c0e13ed9d2585258f278d4bbdd62330accab3cc30a4841" Jul 12 22:11:16.259518 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:16.259489 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jx469_openshift-console-operator(1dcff9c3-da5f-47d9-a082-7a79e9adad93)\"" pod="openshift-console-operator/console-operator-7866497494-jx469" podUID="1dcff9c3-da5f-47d9-a082-7a79e9adad93" Jul 12 22:11:16.949662 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:16.949553 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr" event={"ID":"47b707c0-46ba-4dab-8d0c-311ffc7d7e6e","Type":"ContainerStarted","Data":"eb3de81d1909345e6371ddfe93e57fff8929bc3c1a2e906607f85938d48650fd"} Jul 12 22:11:16.949662 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:16.949628 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr" event={"ID":"47b707c0-46ba-4dab-8d0c-311ffc7d7e6e","Type":"ContainerStarted","Data":"7848bbe2e497b53ce7c6775c98d76b674d690dae2a2fa963249052d48e30b8eb"} Jul 12 22:11:16.974797 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:16.974741 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-nblhr" podStartSLOduration=2.860486603 podStartE2EDuration="3.974725849s" totalImagesPullingTime="1.114239246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:16.973728497 +0000 UTC m=+58.842339320" watchObservedRunningTime="2026-07-12 22:11:16.974725849 +0000 UTC m=+58.843336672" Jul 12 22:11:17.068880 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:17.068847 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:17.166716 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:17.166681 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:17.313359 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:17.313263 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:17.419381 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:17.419330 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:18.016268 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:18.016227 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:18.140236 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:18.140205 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:18.554614 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:18.554585 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:18.834041 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:18.833962 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:18.834041 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:18.834018 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:18.834213 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:18.834101 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 22:11:18.834213 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:18.834104 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 22:11:18.834213 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:18.834161 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:26.834144573 +0000 UTC m=+68.702755389 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : configmap "kube-rbac-proxy" not found Jul 12 22:11:18.834213 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:18.834175 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls podName:9842646b-932e-4847-8502-d2ecbad5e029 nodeName:}" failed. No retries permitted until 2026-07-12 22:11:26.834169159 +0000 UTC m=+68.702779961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcx4l" (UID: "9842646b-932e-4847-8502-d2ecbad5e029") : secret "insights-runtime-extractor-tls" not found Jul 12 22:11:18.960589 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:18.960556 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:19.169508 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:19.169454 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:19.568427 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:19.568342 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:19.842930 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:19.842902 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvgbg" Jul 12 22:11:19.982839 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:19.982779 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:20.350794 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:20.350748 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:20.549677 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:20.549636 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:21.011064 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:21.011012 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:21.081675 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:21.081644 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:21.153264 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:21.153233 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:21.754599 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:21.754564 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:21.857257 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:21.857216 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:22.151562 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:22.151524 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:22.359353 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:22.359319 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:22.752218 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:22.752184 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:23.170362 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:23.170327 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:23.757411 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:23.757357 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:23.834021 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:23.833991 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:23.960174 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:23.960142 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:24.364882 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.364847 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:24.378944 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.378921 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:11:24.381840 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.381820 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 22:11:24.389289 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:24.389271 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 22:11:24.389390 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:11:24.389322 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs podName:71466df1-3f21-4c6a-acbe-758aeea5531d nodeName:}" failed. No retries permitted until 2026-07-12 22:12:28.389306177 +0000 UTC m=+130.257916979 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs") pod "network-metrics-daemon-gq8dd" (UID: "71466df1-3f21-4c6a-acbe-758aeea5531d") : secret "metrics-daemon-secret" not found Jul 12 22:11:24.480039 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.480004 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:11:24.482747 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.482731 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 22:11:24.493349 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.493330 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87116643-5000-4846-a88b-a452ab0ef89e-original-pull-secret\") pod \"global-pull-secret-syncer-cd9nz\" (UID: \"87116643-5000-4846-a88b-a452ab0ef89e\") " pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:11:24.557088 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.557063 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cd9nz" Jul 12 22:11:24.581154 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.581124 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:11:24.583997 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.583976 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 22:11:24.595157 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.595133 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 22:11:24.606786 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.606742 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n95n7\" (UniqueName: \"kubernetes.io/projected/957fa4c9-20c3-4524-9b3d-4ed86dbda907-kube-api-access-n95n7\") pod \"network-check-target-pzppf\" (UID: \"957fa4c9-20c3-4524-9b3d-4ed86dbda907\") " pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:11:24.680802 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.680770 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cd9nz"] Jul 12 22:11:24.684441 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:24.684416 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87116643_5000_4846_a88b_a452ab0ef89e.slice/crio-cca6f194cbd7076d97c51f1283a16b3cea01681d95397cbec011c806d42b1ba6 WatchSource:0}: Error finding container cca6f194cbd7076d97c51f1283a16b3cea01681d95397cbec011c806d42b1ba6: Status 404 returned error can't find the container with id cca6f194cbd7076d97c51f1283a16b3cea01681d95397cbec011c806d42b1ba6 Jul 12 22:11:24.744543 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.744517 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:24.846345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.846314 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bgjrf\"" Jul 12 22:11:24.853924 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.853900 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:11:24.954570 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.954400 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:24.977719 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:24.977684 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cd9nz" event={"ID":"87116643-5000-4846-a88b-a452ab0ef89e","Type":"ContainerStarted","Data":"cca6f194cbd7076d97c51f1283a16b3cea01681d95397cbec011c806d42b1ba6"} Jul 12 22:11:25.003390 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.003342 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pzppf"] Jul 12 22:11:25.006230 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:25.006207 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957fa4c9_20c3_4524_9b3d_4ed86dbda907.slice/crio-09c731693d8f27caac389c003ed48633112dded4058d63809187a646d1bfe30b WatchSource:0}: Error finding container 09c731693d8f27caac389c003ed48633112dded4058d63809187a646d1bfe30b: Status 404 returned error can't find the container with id 09c731693d8f27caac389c003ed48633112dded4058d63809187a646d1bfe30b Jul 12 22:11:25.357677 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.357623 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:25.387727 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.387694 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:11:25.390667 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.390640 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"image-registry-649755d64-dr2bt\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:11:25.488480 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.488439 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:11:25.491263 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.491235 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80c47a8b-861f-4216-98fc-ea06a6de52f7-metrics-tls\") pod \"dns-default-wlkph\" (UID: \"80c47a8b-861f-4216-98fc-ea06a6de52f7\") " pod="openshift-dns/dns-default-wlkph" Jul 12 22:11:25.630840 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.630766 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hjmtz\"" Jul 12 22:11:25.638391 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.638342 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:11:25.737346 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.737318 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9r9hf\"" Jul 12 22:11:25.745769 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.745740 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wlkph" Jul 12 22:11:25.778078 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.778021 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:25.828643 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.828565 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-649755d64-dr2bt"] Jul 12 22:11:25.840178 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:25.840144 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ef56a7_6caf_4ed6_b5f4_dc446ac67697.slice/crio-a4e5178064a7497297404fa17417e896051c10d5b77a3dbf9ff048cfa3c2f368 WatchSource:0}: Error finding container a4e5178064a7497297404fa17417e896051c10d5b77a3dbf9ff048cfa3c2f368: Status 404 returned error can't find the container with id a4e5178064a7497297404fa17417e896051c10d5b77a3dbf9ff048cfa3c2f368 Jul 12 22:11:25.934872 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.934844 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wlkph"] Jul 12 22:11:25.938461 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:25.938433 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c47a8b_861f_4216_98fc_ea06a6de52f7.slice/crio-3e796b28d22f83b853b14fdeb576e7fe0196edd5dff453715c3a59b388af0e0b WatchSource:0}: Error finding container 3e796b28d22f83b853b14fdeb576e7fe0196edd5dff453715c3a59b388af0e0b: Status 404 returned error can't find the container with id 3e796b28d22f83b853b14fdeb576e7fe0196edd5dff453715c3a59b388af0e0b Jul 12 22:11:25.964799 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.963405 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" limit=1024 Jul 12 22:11:25.987320 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.986731 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649755d64-dr2bt" event={"ID":"27ef56a7-6caf-4ed6-b5f4-dc446ac67697","Type":"ContainerStarted","Data":"71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5"} Jul 12 22:11:25.987320 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.986788 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649755d64-dr2bt" event={"ID":"27ef56a7-6caf-4ed6-b5f4-dc446ac67697","Type":"ContainerStarted","Data":"a4e5178064a7497297404fa17417e896051c10d5b77a3dbf9ff048cfa3c2f368"} Jul 12 22:11:25.990217 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.989879 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pzppf" event={"ID":"957fa4c9-20c3-4524-9b3d-4ed86dbda907","Type":"ContainerStarted","Data":"09c731693d8f27caac389c003ed48633112dded4058d63809187a646d1bfe30b"} Jul 12 22:11:25.993082 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:25.992968 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wlkph" event={"ID":"80c47a8b-861f-4216-98fc-ea06a6de52f7","Type":"ContainerStarted","Data":"3e796b28d22f83b853b14fdeb576e7fe0196edd5dff453715c3a59b388af0e0b"} Jul 12 22:11:26.900442 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:26.900405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:26.900891 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:26.900487 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:26.900984 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:26.900966 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9842646b-932e-4847-8502-d2ecbad5e029-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:26.905045 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:26.905018 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9842646b-932e-4847-8502-d2ecbad5e029-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcx4l\" (UID: \"9842646b-932e-4847-8502-d2ecbad5e029\") " pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:26.959071 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:26.958936 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5sgr4\"" Jul 12 22:11:26.966226 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:26.966194 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zcx4l" Jul 12 22:11:26.999969 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:26.999942 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:11:27.029035 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:27.028394 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-649755d64-dr2bt" podStartSLOduration=68.028346849 podStartE2EDuration="1m8.028346849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:27.026603123 +0000 UTC m=+68.895213948" watchObservedRunningTime="2026-07-12 22:11:27.028346849 +0000 UTC m=+68.896957673" Jul 12 22:11:27.158030 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:27.157944 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zcx4l"] Jul 12 22:11:27.161174 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:27.161136 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9842646b_932e_4847_8502_d2ecbad5e029.slice/crio-205c9ae97d1bf590810e57b03ac97474d9a98b2cd19b703c8fe730f9937cf148 WatchSource:0}: Error finding container 205c9ae97d1bf590810e57b03ac97474d9a98b2cd19b703c8fe730f9937cf148: Status 404 returned error can't find the container with id 205c9ae97d1bf590810e57b03ac97474d9a98b2cd19b703c8fe730f9937cf148 Jul 12 22:11:28.006988 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:28.006940 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcx4l" event={"ID":"9842646b-932e-4847-8502-d2ecbad5e029","Type":"ContainerStarted","Data":"205c9ae97d1bf590810e57b03ac97474d9a98b2cd19b703c8fe730f9937cf148"} Jul 12 22:11:30.626012 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:30.625981 2563 scope.go:117] "RemoveContainer" containerID="cefeebf7e795275378c0e13ed9d2585258f278d4bbdd62330accab3cc30a4841" Jul 12 22:11:31.021613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:31.021544 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:11:31.021780 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:31.021640 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jx469" event={"ID":"1dcff9c3-da5f-47d9-a082-7a79e9adad93","Type":"ContainerStarted","Data":"48355fa0881a5c875554e75bb72e85da55518ede918d4d343d7ad199d3d6e953"} Jul 12 22:11:31.022349 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:31.022271 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:31.041069 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:31.041014 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-jx469" podStartSLOduration=22.221582328 podStartE2EDuration="26.040996554s" totalImagesPullingTime="3.819414226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:31.040903155 +0000 UTC m=+72.909513979" watchObservedRunningTime="2026-07-12 22:11:31.040996554 +0000 UTC m=+72.909607379" Jul 12 22:11:31.504104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:31.504038 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-jx469" Jul 12 22:11:33.030479 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.030399 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wlkph" event={"ID":"80c47a8b-861f-4216-98fc-ea06a6de52f7","Type":"ContainerStarted","Data":"3f0bb91c6cd84d3320b5efe295f1e1a90f52a0a445ac13fa8162ab3788ef76e9"} Jul 12 22:11:33.594723 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.594690 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-clt97"] Jul 12 22:11:33.597765 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.597743 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-clt97" Jul 12 22:11:33.600098 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.600075 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 12 22:11:33.601031 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.601006 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 12 22:11:33.601156 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.601140 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jnz97\"" Jul 12 22:11:33.617842 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.614945 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-clt97"] Jul 12 22:11:33.658383 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.658337 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdgwh\" (UniqueName: \"kubernetes.io/projected/b6a3d700-67f4-4928-be3a-89f5419a8eb5-kube-api-access-fdgwh\") pod \"downloads-68c9cbb497-clt97\" (UID: \"b6a3d700-67f4-4928-be3a-89f5419a8eb5\") " pod="openshift-console/downloads-68c9cbb497-clt97" Jul 12 22:11:33.758960 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.758932 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdgwh\" (UniqueName: \"kubernetes.io/projected/b6a3d700-67f4-4928-be3a-89f5419a8eb5-kube-api-access-fdgwh\") pod \"downloads-68c9cbb497-clt97\" (UID: \"b6a3d700-67f4-4928-be3a-89f5419a8eb5\") " pod="openshift-console/downloads-68c9cbb497-clt97" Jul 12 22:11:33.770710 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.770665 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdgwh\" (UniqueName: \"kubernetes.io/projected/b6a3d700-67f4-4928-be3a-89f5419a8eb5-kube-api-access-fdgwh\") pod \"downloads-68c9cbb497-clt97\" (UID: \"b6a3d700-67f4-4928-be3a-89f5419a8eb5\") " pod="openshift-console/downloads-68c9cbb497-clt97" Jul 12 22:11:33.909545 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:33.909462 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-clt97" Jul 12 22:11:34.035489 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.035458 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cd9nz" event={"ID":"87116643-5000-4846-a88b-a452ab0ef89e","Type":"ContainerStarted","Data":"85600f172f5445f5461aba81cd55ef2d785a9ad9f15a3da208ab04365558b5fb"} Jul 12 22:11:34.036994 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.036972 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pzppf" event={"ID":"957fa4c9-20c3-4524-9b3d-4ed86dbda907","Type":"ContainerStarted","Data":"22d5a57da014ec378221a7617cfc7a4621091925f2433adffc0039c820640c2d"} Jul 12 22:11:34.037087 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.037015 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:11:34.038897 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.038874 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcx4l" event={"ID":"9842646b-932e-4847-8502-d2ecbad5e029","Type":"ContainerStarted","Data":"ec79c492e4f99528c50734880b169fcaaef79203e4998cbab0c9bd08b282ae26"} Jul 12 22:11:34.038997 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.038924 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcx4l" event={"ID":"9842646b-932e-4847-8502-d2ecbad5e029","Type":"ContainerStarted","Data":"5b1bb65a08de57cc4e8c4a4d4a0779d51f52432d56cf79e0cc290a255d9f6f43"} Jul 12 22:11:34.040753 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.040733 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wlkph" event={"ID":"80c47a8b-861f-4216-98fc-ea06a6de52f7","Type":"ContainerStarted","Data":"3f93fdf198c0618937462ba689a42746c87d03cd008ffeebf7c26effeacf8837"} Jul 12 22:11:34.040879 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.040863 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wlkph" Jul 12 22:11:34.063757 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.063736 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-clt97"] Jul 12 22:11:34.067414 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:34.067392 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a3d700_67f4_4928_be3a_89f5419a8eb5.slice/crio-639c50c432f25fd6b4a02e446e6ff8d9ff7df6d2fbf1068cbb1fc78c67f842d6 WatchSource:0}: Error finding container 639c50c432f25fd6b4a02e446e6ff8d9ff7df6d2fbf1068cbb1fc78c67f842d6: Status 404 returned error can't find the container with id 639c50c432f25fd6b4a02e446e6ff8d9ff7df6d2fbf1068cbb1fc78c67f842d6 Jul 12 22:11:34.075287 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.075246 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cd9nz" podStartSLOduration=67.655725401 podStartE2EDuration="1m15.075233901s" totalImagesPullingTime="7.4195085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:34.073755808 +0000 UTC m=+75.942366632" watchObservedRunningTime="2026-07-12 22:11:34.075233901 +0000 UTC m=+75.943844724" Jul 12 22:11:34.099475 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.099432 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pzppf" podStartSLOduration=69.394352109 podStartE2EDuration="1m15.099420286s" totalImagesPullingTime="5.705068177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:34.09867318 +0000 UTC m=+75.967284015" watchObservedRunningTime="2026-07-12 22:11:34.099420286 +0000 UTC m=+75.968031088" Jul 12 22:11:34.121206 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:34.121165 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wlkph" podStartSLOduration=36.35498734 podStartE2EDuration="41.12115322s" totalImagesPullingTime="4.76616588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:10:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:34.120837757 +0000 UTC m=+75.989448580" watchObservedRunningTime="2026-07-12 22:11:34.12115322 +0000 UTC m=+75.989764035" Jul 12 22:11:35.046992 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:35.046906 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-clt97" event={"ID":"b6a3d700-67f4-4928-be3a-89f5419a8eb5","Type":"ContainerStarted","Data":"639c50c432f25fd6b4a02e446e6ff8d9ff7df6d2fbf1068cbb1fc78c67f842d6"} Jul 12 22:11:37.056790 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:37.056759 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcx4l" event={"ID":"9842646b-932e-4847-8502-d2ecbad5e029","Type":"ContainerStarted","Data":"938faa19bfbf954bbe97a249d61b976fcf1c66889da775f10b6959b3706bec81"} Jul 12 22:11:37.078513 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:37.078456 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zcx4l" podStartSLOduration=17.846145501 podStartE2EDuration="27.078434663s" totalImagesPullingTime="9.232289162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:37.076763454 +0000 UTC m=+78.945374277" watchObservedRunningTime="2026-07-12 22:11:37.078434663 +0000 UTC m=+78.947045488" Jul 12 22:11:44.050690 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:44.050662 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wlkph" Jul 12 22:11:48.012309 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:48.012280 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:11:52.116109 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:52.116072 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-clt97" event={"ID":"b6a3d700-67f4-4928-be3a-89f5419a8eb5","Type":"ContainerStarted","Data":"2e68d10c6118959b0a5b616b8838ce4a2f65bd417090a074855171360e291a5f"} Jul 12 22:11:52.116715 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:52.116691 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-clt97" Jul 12 22:11:52.138804 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:52.138743 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-clt97" podStartSLOduration=1.6878734340000001 podStartE2EDuration="19.138725043s" totalImagesPullingTime="17.450851609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:11:52.13633972 +0000 UTC m=+94.004950543" watchObservedRunningTime="2026-07-12 22:11:52.138725043 +0000 UTC m=+94.007335867" Jul 12 22:11:52.139186 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:52.139125 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-clt97" Jul 12 22:11:57.716405 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.716320 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mgfgq"] Jul 12 22:11:57.746810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.746775 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.750399 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.750347 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 22:11:57.750733 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.750461 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 22:11:57.750871 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.750858 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k7pht\"" Jul 12 22:11:57.751133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.751112 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 22:11:57.751472 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.751283 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 22:11:57.751472 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.751323 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 22:11:57.751472 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.751344 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 22:11:57.803896 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.803859 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-649755d64-dr2bt"] Jul 12 22:11:57.868255 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.868179 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-textfile\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.868255 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.868247 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skkhq\" (UniqueName: \"kubernetes.io/projected/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-kube-api-access-skkhq\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.868514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.868291 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-sys\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.868514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.868311 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-tls\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.868514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.868334 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.868514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.868361 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.868514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.868407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-metrics-client-ca\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.868514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.868434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-root\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.868514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.868460 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-wtmp\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969314 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skkhq\" (UniqueName: \"kubernetes.io/projected/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-kube-api-access-skkhq\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969550 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969389 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-sys\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969550 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969421 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-tls\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969550 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969445 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969550 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969550 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969467 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-sys\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969550 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969497 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-metrics-client-ca\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969550 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969548 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-root\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969923 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969593 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-root\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969923 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969675 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-wtmp\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969923 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-textfile\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.969923 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.969876 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-wtmp\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.970179 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.970151 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-accelerators-collector-config\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.970352 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.970306 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-textfile\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.970581 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.970561 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-metrics-client-ca\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.972414 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.972390 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.972508 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.972491 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-node-exporter-tls\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:57.984188 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:57.984161 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skkhq\" (UniqueName: \"kubernetes.io/projected/67fb80af-f51e-4c1e-8c22-a6b2869e0f9e-kube-api-access-skkhq\") pod \"node-exporter-mgfgq\" (UID: \"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e\") " pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:58.059155 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:58.059125 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mgfgq" Jul 12 22:11:58.086797 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:11:58.086760 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67fb80af_f51e_4c1e_8c22_a6b2869e0f9e.slice/crio-00b5f2ec2513a6896896a12fdc766b51dd88a35bc6ac824dd36b704513f0d17a WatchSource:0}: Error finding container 00b5f2ec2513a6896896a12fdc766b51dd88a35bc6ac824dd36b704513f0d17a: Status 404 returned error can't find the container with id 00b5f2ec2513a6896896a12fdc766b51dd88a35bc6ac824dd36b704513f0d17a Jul 12 22:11:58.146052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:11:58.145974 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgfgq" event={"ID":"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e","Type":"ContainerStarted","Data":"00b5f2ec2513a6896896a12fdc766b51dd88a35bc6ac824dd36b704513f0d17a"} Jul 12 22:12:00.156152 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:00.156070 2563 generic.go:356] "Generic (PLEG): container finished" podID="67fb80af-f51e-4c1e-8c22-a6b2869e0f9e" containerID="53d2589fb040651b52f91753ee27e9e00152c40709b1f6131c4f362f0f7beafc" exitCode=0 Jul 12 22:12:00.156611 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:00.156160 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgfgq" event={"ID":"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e","Type":"ContainerDied","Data":"53d2589fb040651b52f91753ee27e9e00152c40709b1f6131c4f362f0f7beafc"} Jul 12 22:12:01.161981 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:01.161945 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgfgq" event={"ID":"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e","Type":"ContainerStarted","Data":"0ecebeea5c6a3d89426e75339aa08e65b272bff801f95a622401f0061684c833"} Jul 12 22:12:01.162465 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:01.161999 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgfgq" event={"ID":"67fb80af-f51e-4c1e-8c22-a6b2869e0f9e","Type":"ContainerStarted","Data":"a2d5d69e0fb0fe5aa543e07f2bf8056610c91ae8f100cc22dcfd0f2c9e669ab7"} Jul 12 22:12:01.183913 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:01.183862 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mgfgq" podStartSLOduration=2.528388901 podStartE2EDuration="4.183847821s" totalImagesPullingTime="1.544808557s" totalInitContainerRuntime="110.650363ms" isStatefulPod=true podCreationTimestamp="2026-07-12 22:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:01.181769244 +0000 UTC m=+103.050380073" watchObservedRunningTime="2026-07-12 22:12:01.183847821 +0000 UTC m=+103.052458644" Jul 12 22:12:05.049280 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:05.049254 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pzppf" Jul 12 22:12:21.232253 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:21.232216 2563 generic.go:356] "Generic (PLEG): container finished" podID="d3164201-3898-4439-907f-3f9d20dd5c57" containerID="f4ab967e5866d64dd6eb496e9510a70585acc9dee6320bdb6b7419d4e67f32eb" exitCode=0 Jul 12 22:12:21.232778 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:21.232269 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" event={"ID":"d3164201-3898-4439-907f-3f9d20dd5c57","Type":"ContainerDied","Data":"f4ab967e5866d64dd6eb496e9510a70585acc9dee6320bdb6b7419d4e67f32eb"} Jul 12 22:12:21.232778 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:21.232676 2563 scope.go:117] "RemoveContainer" containerID="f4ab967e5866d64dd6eb496e9510a70585acc9dee6320bdb6b7419d4e67f32eb" Jul 12 22:12:22.237619 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:22.237585 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-wtzwb" event={"ID":"d3164201-3898-4439-907f-3f9d20dd5c57","Type":"ContainerStarted","Data":"eed3f57ba3d399701b4304613c30e1781aff831230bf833ee41f2110751fb45e"} Jul 12 22:12:22.834960 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:22.834919 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-649755d64-dr2bt" podUID="27ef56a7-6caf-4ed6-b5f4-dc446ac67697" containerName="registry" containerID="cri-o://71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5" gracePeriod=30 Jul 12 22:12:23.116464 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.116435 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:12:23.242243 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.242205 2563 generic.go:356] "Generic (PLEG): container finished" podID="27ef56a7-6caf-4ed6-b5f4-dc446ac67697" containerID="71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5" exitCode=0 Jul 12 22:12:23.242652 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.242256 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649755d64-dr2bt" Jul 12 22:12:23.242652 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.242290 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649755d64-dr2bt" event={"ID":"27ef56a7-6caf-4ed6-b5f4-dc446ac67697","Type":"ContainerDied","Data":"71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5"} Jul 12 22:12:23.242652 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.242344 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649755d64-dr2bt" event={"ID":"27ef56a7-6caf-4ed6-b5f4-dc446ac67697","Type":"ContainerDied","Data":"a4e5178064a7497297404fa17417e896051c10d5b77a3dbf9ff048cfa3c2f368"} Jul 12 22:12:23.242652 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.242389 2563 scope.go:117] "RemoveContainer" containerID="71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5" Jul 12 22:12:23.250334 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.250315 2563 scope.go:117] "RemoveContainer" containerID="71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5" Jul 12 22:12:23.250593 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:12:23.250573 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5\": container with ID starting with 71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5 not found: ID does not exist" containerID="71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5" Jul 12 22:12:23.250648 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.250600 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5"} err="failed to get container status \"71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5\": rpc error: code = NotFound desc = could not find container \"71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5\": container with ID starting with 71dbf8eff200fd144f0f9b48c085b41194b0147ea177817758b8f7b8a5b71af5 not found: ID does not exist" Jul 12 22:12:23.277884 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.277863 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-trusted-ca\") pod \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " Jul 12 22:12:23.277972 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.277916 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-image-registry-private-configuration\") pod \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " Jul 12 22:12:23.277972 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.277935 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-bound-sa-token\") pod \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " Jul 12 22:12:23.277972 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.277954 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-certificates\") pod \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " Jul 12 22:12:23.278119 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.277976 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmgss\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-kube-api-access-lmgss\") pod \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " Jul 12 22:12:23.278119 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.277997 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-ca-trust-extracted\") pod \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " Jul 12 22:12:23.278119 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.278095 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") pod \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " Jul 12 22:12:23.278264 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.278144 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-installation-pull-secrets\") pod \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\" (UID: \"27ef56a7-6caf-4ed6-b5f4-dc446ac67697\") " Jul 12 22:12:23.278347 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.278322 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "27ef56a7-6caf-4ed6-b5f4-dc446ac67697" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:12:23.278453 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.278425 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "27ef56a7-6caf-4ed6-b5f4-dc446ac67697" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:12:23.278507 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.278479 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-trusted-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:12:23.280757 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.280712 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-kube-api-access-lmgss" (OuterVolumeSpecName: "kube-api-access-lmgss") pod "27ef56a7-6caf-4ed6-b5f4-dc446ac67697" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697"). InnerVolumeSpecName "kube-api-access-lmgss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:12:23.280872 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.280782 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "27ef56a7-6caf-4ed6-b5f4-dc446ac67697" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:12:23.280932 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.280847 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "27ef56a7-6caf-4ed6-b5f4-dc446ac67697" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:12:23.280932 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.280916 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "27ef56a7-6caf-4ed6-b5f4-dc446ac67697" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:12:23.281017 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.281002 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "27ef56a7-6caf-4ed6-b5f4-dc446ac67697" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:12:23.287156 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.287135 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "27ef56a7-6caf-4ed6-b5f4-dc446ac67697" (UID: "27ef56a7-6caf-4ed6-b5f4-dc446ac67697"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:12:23.379613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.379542 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-image-registry-private-configuration\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:12:23.379613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.379569 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-bound-sa-token\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:12:23.379613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.379578 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-certificates\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:12:23.379613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.379586 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmgss\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-kube-api-access-lmgss\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:12:23.379613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.379594 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-ca-trust-extracted\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:12:23.379613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.379602 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-registry-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:12:23.379613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.379610 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27ef56a7-6caf-4ed6-b5f4-dc446ac67697-installation-pull-secrets\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:12:23.563024 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.562998 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-649755d64-dr2bt"] Jul 12 22:12:23.568674 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:23.568652 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-649755d64-dr2bt"] Jul 12 22:12:24.629941 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:24.629908 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27ef56a7-6caf-4ed6-b5f4-dc446ac67697" path="/var/lib/kubelet/pods/27ef56a7-6caf-4ed6-b5f4-dc446ac67697/volumes" Jul 12 22:12:28.411281 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:28.411235 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:12:28.413808 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:28.413785 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71466df1-3f21-4c6a-acbe-758aeea5531d-metrics-certs\") pod \"network-metrics-daemon-gq8dd\" (UID: \"71466df1-3f21-4c6a-acbe-758aeea5531d\") " pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:12:28.452929 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:28.452894 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-npkdh\"" Jul 12 22:12:28.460026 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:28.460004 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gq8dd" Jul 12 22:12:28.639556 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:28.639514 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gq8dd"] Jul 12 22:12:28.642814 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:12:28.642789 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71466df1_3f21_4c6a_acbe_758aeea5531d.slice/crio-7221cb7cef3b8483a4259a3dc08c591718da4b2d0f3885719fbc18e0f3ff1fc8 WatchSource:0}: Error finding container 7221cb7cef3b8483a4259a3dc08c591718da4b2d0f3885719fbc18e0f3ff1fc8: Status 404 returned error can't find the container with id 7221cb7cef3b8483a4259a3dc08c591718da4b2d0f3885719fbc18e0f3ff1fc8 Jul 12 22:12:29.266605 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:29.266562 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gq8dd" event={"ID":"71466df1-3f21-4c6a-acbe-758aeea5531d","Type":"ContainerStarted","Data":"7221cb7cef3b8483a4259a3dc08c591718da4b2d0f3885719fbc18e0f3ff1fc8"} Jul 12 22:12:30.272169 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:30.272099 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gq8dd" event={"ID":"71466df1-3f21-4c6a-acbe-758aeea5531d","Type":"ContainerStarted","Data":"e6ae1b1fd4d4264af9669441a4b0adb9cb2a64abddba88202bc3167453818e13"} Jul 12 22:12:31.277136 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:31.277097 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gq8dd" event={"ID":"71466df1-3f21-4c6a-acbe-758aeea5531d","Type":"ContainerStarted","Data":"29b08468e8b535041ea8c71a0902ab2539141bfaaf5672425df2c918d7005517"} Jul 12 22:12:31.295541 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:31.295493 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gq8dd" podStartSLOduration=131.861909085 podStartE2EDuration="2m13.295477903s" totalImagesPullingTime="1.433568818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:31.294328602 +0000 UTC m=+133.162939427" watchObservedRunningTime="2026-07-12 22:12:31.295477903 +0000 UTC m=+133.164088746" Jul 12 22:12:41.314310 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:41.314226 2563 generic.go:356] "Generic (PLEG): container finished" podID="cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf" containerID="22e38599ab3123a1105be8872c786610000e39ce37f224e7beded896319d8acf" exitCode=0 Jul 12 22:12:41.314692 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:41.314311 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" event={"ID":"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf","Type":"ContainerDied","Data":"22e38599ab3123a1105be8872c786610000e39ce37f224e7beded896319d8acf"} Jul 12 22:12:41.314732 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:41.314715 2563 scope.go:117] "RemoveContainer" containerID="22e38599ab3123a1105be8872c786610000e39ce37f224e7beded896319d8acf" Jul 12 22:12:42.320260 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:12:42.320229 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-s9vnd" event={"ID":"cb95c7eb-6ba2-49d9-a3e9-5b6eea580fcf","Type":"ContainerStarted","Data":"a2ec519b7b7f1676f9dd184d2ef80736e1e792c39ce698bb0fdd44f1e4388c40"} Jul 12 22:14:10.062362 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.062329 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f"] Jul 12 22:14:10.062867 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.062652 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27ef56a7-6caf-4ed6-b5f4-dc446ac67697" containerName="registry" Jul 12 22:14:10.062867 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.062664 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="27ef56a7-6caf-4ed6-b5f4-dc446ac67697" containerName="registry" Jul 12 22:14:10.062867 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.062726 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="27ef56a7-6caf-4ed6-b5f4-dc446ac67697" containerName="registry" Jul 12 22:14:10.064685 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.064658 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.067261 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.067235 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4prg8\"" Jul 12 22:14:10.067261 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.067262 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 22:14:10.068085 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.068070 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 22:14:10.077875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.077853 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f"] Jul 12 22:14:10.133586 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.133551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.133586 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.133591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.133775 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.133683 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrn5n\" (UniqueName: \"kubernetes.io/projected/81a20066-0d72-4a1a-8dd9-b587714f9dfe-kube-api-access-vrn5n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.234454 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.234424 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrn5n\" (UniqueName: \"kubernetes.io/projected/81a20066-0d72-4a1a-8dd9-b587714f9dfe-kube-api-access-vrn5n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.234594 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.234458 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.234594 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.234480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.234803 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.234783 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.234837 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.234812 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.243475 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.243446 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrn5n\" (UniqueName: \"kubernetes.io/projected/81a20066-0d72-4a1a-8dd9-b587714f9dfe-kube-api-access-vrn5n\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.373580 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.373538 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:10.500104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.500077 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f"] Jul 12 22:14:10.502604 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:14:10.502582 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a20066_0d72_4a1a_8dd9_b587714f9dfe.slice/crio-8a1b999d836ad2b55a1c44a8b85c6ca697ae2470d1155b618c50facf52d66be8 WatchSource:0}: Error finding container 8a1b999d836ad2b55a1c44a8b85c6ca697ae2470d1155b618c50facf52d66be8: Status 404 returned error can't find the container with id 8a1b999d836ad2b55a1c44a8b85c6ca697ae2470d1155b618c50facf52d66be8 Jul 12 22:14:10.618940 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:10.618905 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" event={"ID":"81a20066-0d72-4a1a-8dd9-b587714f9dfe","Type":"ContainerStarted","Data":"8a1b999d836ad2b55a1c44a8b85c6ca697ae2470d1155b618c50facf52d66be8"} Jul 12 22:14:16.639129 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:16.639098 2563 generic.go:356] "Generic (PLEG): container finished" podID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerID="cf8f8e3137816c58105413ee5579335a3b50eb62c9f77954d87be44aca19f0da" exitCode=0 Jul 12 22:14:16.639468 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:16.639186 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" event={"ID":"81a20066-0d72-4a1a-8dd9-b587714f9dfe","Type":"ContainerDied","Data":"cf8f8e3137816c58105413ee5579335a3b50eb62c9f77954d87be44aca19f0da"} Jul 12 22:14:18.648145 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:18.648105 2563 generic.go:356] "Generic (PLEG): container finished" podID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerID="9cefd60368ffde62c1d6c73b33eebe5d1d9c63f676058427080ebcc5b12456b9" exitCode=0 Jul 12 22:14:18.648512 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:18.648188 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" event={"ID":"81a20066-0d72-4a1a-8dd9-b587714f9dfe","Type":"ContainerDied","Data":"9cefd60368ffde62c1d6c73b33eebe5d1d9c63f676058427080ebcc5b12456b9"} Jul 12 22:14:26.678254 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:26.678222 2563 generic.go:356] "Generic (PLEG): container finished" podID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerID="41cb4fd4e4ce5d9d20813508a5e4b25e2776cef1ab2ba9448e4e0dc09d42b333" exitCode=0 Jul 12 22:14:26.678673 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:26.678313 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" event={"ID":"81a20066-0d72-4a1a-8dd9-b587714f9dfe","Type":"ContainerDied","Data":"41cb4fd4e4ce5d9d20813508a5e4b25e2776cef1ab2ba9448e4e0dc09d42b333"} Jul 12 22:14:27.800604 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.800584 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:27.887776 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.887743 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrn5n\" (UniqueName: \"kubernetes.io/projected/81a20066-0d72-4a1a-8dd9-b587714f9dfe-kube-api-access-vrn5n\") pod \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " Jul 12 22:14:27.887951 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.887829 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-util\") pod \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " Jul 12 22:14:27.887951 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.887861 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-bundle\") pod \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\" (UID: \"81a20066-0d72-4a1a-8dd9-b587714f9dfe\") " Jul 12 22:14:27.888550 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.888521 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-bundle" (OuterVolumeSpecName: "bundle") pod "81a20066-0d72-4a1a-8dd9-b587714f9dfe" (UID: "81a20066-0d72-4a1a-8dd9-b587714f9dfe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:14:27.889902 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.889880 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a20066-0d72-4a1a-8dd9-b587714f9dfe-kube-api-access-vrn5n" (OuterVolumeSpecName: "kube-api-access-vrn5n") pod "81a20066-0d72-4a1a-8dd9-b587714f9dfe" (UID: "81a20066-0d72-4a1a-8dd9-b587714f9dfe"). InnerVolumeSpecName "kube-api-access-vrn5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:14:27.891774 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.891754 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-util" (OuterVolumeSpecName: "util") pod "81a20066-0d72-4a1a-8dd9-b587714f9dfe" (UID: "81a20066-0d72-4a1a-8dd9-b587714f9dfe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:14:27.988391 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.988293 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrn5n\" (UniqueName: \"kubernetes.io/projected/81a20066-0d72-4a1a-8dd9-b587714f9dfe-kube-api-access-vrn5n\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:14:27.988391 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.988321 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-util\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:14:27.988391 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:27.988330 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81a20066-0d72-4a1a-8dd9-b587714f9dfe-bundle\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:14:28.686751 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:28.686724 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" Jul 12 22:14:28.686897 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:28.686727 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfw8f" event={"ID":"81a20066-0d72-4a1a-8dd9-b587714f9dfe","Type":"ContainerDied","Data":"8a1b999d836ad2b55a1c44a8b85c6ca697ae2470d1155b618c50facf52d66be8"} Jul 12 22:14:28.686897 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:28.686831 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a1b999d836ad2b55a1c44a8b85c6ca697ae2470d1155b618c50facf52d66be8" Jul 12 22:14:33.181945 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.181916 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6"] Jul 12 22:14:33.182345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.182163 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerName="extract" Jul 12 22:14:33.182345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.182173 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerName="extract" Jul 12 22:14:33.182345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.182183 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerName="util" Jul 12 22:14:33.182345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.182189 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerName="util" Jul 12 22:14:33.182345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.182196 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerName="pull" Jul 12 22:14:33.182345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.182201 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerName="pull" Jul 12 22:14:33.182345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.182251 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a20066-0d72-4a1a-8dd9-b587714f9dfe" containerName="extract" Jul 12 22:14:33.220788 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.220756 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6"] Jul 12 22:14:33.220904 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.220840 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" Jul 12 22:14:33.223711 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.223692 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 12 22:14:33.223885 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.223871 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gr92k\"" Jul 12 22:14:33.223957 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.223903 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 12 22:14:33.326118 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.326092 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd44f3b6-7cdb-4729-b582-ad87b1b16f9f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9qld6\" (UID: \"bd44f3b6-7cdb-4729-b582-ad87b1b16f9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" Jul 12 22:14:33.326118 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.326128 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gds8r\" (UniqueName: \"kubernetes.io/projected/bd44f3b6-7cdb-4729-b582-ad87b1b16f9f-kube-api-access-gds8r\") pod \"cert-manager-operator-controller-manager-55c444bf86-9qld6\" (UID: \"bd44f3b6-7cdb-4729-b582-ad87b1b16f9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" Jul 12 22:14:33.427119 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.427091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd44f3b6-7cdb-4729-b582-ad87b1b16f9f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9qld6\" (UID: \"bd44f3b6-7cdb-4729-b582-ad87b1b16f9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" Jul 12 22:14:33.427245 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.427125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gds8r\" (UniqueName: \"kubernetes.io/projected/bd44f3b6-7cdb-4729-b582-ad87b1b16f9f-kube-api-access-gds8r\") pod \"cert-manager-operator-controller-manager-55c444bf86-9qld6\" (UID: \"bd44f3b6-7cdb-4729-b582-ad87b1b16f9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" Jul 12 22:14:33.427504 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.427482 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd44f3b6-7cdb-4729-b582-ad87b1b16f9f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9qld6\" (UID: \"bd44f3b6-7cdb-4729-b582-ad87b1b16f9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" Jul 12 22:14:33.437491 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.437436 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gds8r\" (UniqueName: \"kubernetes.io/projected/bd44f3b6-7cdb-4729-b582-ad87b1b16f9f-kube-api-access-gds8r\") pod \"cert-manager-operator-controller-manager-55c444bf86-9qld6\" (UID: \"bd44f3b6-7cdb-4729-b582-ad87b1b16f9f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" Jul 12 22:14:33.529308 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.529282 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" Jul 12 22:14:33.685809 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.685777 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6"] Jul 12 22:14:33.689837 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:14:33.689784 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd44f3b6_7cdb_4729_b582_ad87b1b16f9f.slice/crio-a21f1cfeee8503e418e04e1a3e02a5da37dda4a1e1ab0d0a05aeab0484206f52 WatchSource:0}: Error finding container a21f1cfeee8503e418e04e1a3e02a5da37dda4a1e1ab0d0a05aeab0484206f52: Status 404 returned error can't find the container with id a21f1cfeee8503e418e04e1a3e02a5da37dda4a1e1ab0d0a05aeab0484206f52 Jul 12 22:14:33.705254 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:33.705230 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" event={"ID":"bd44f3b6-7cdb-4729-b582-ad87b1b16f9f","Type":"ContainerStarted","Data":"a21f1cfeee8503e418e04e1a3e02a5da37dda4a1e1ab0d0a05aeab0484206f52"} Jul 12 22:14:36.717682 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:36.717647 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" event={"ID":"bd44f3b6-7cdb-4729-b582-ad87b1b16f9f","Type":"ContainerStarted","Data":"2204a26aa91790021079b92c79d47ccb712377298bec826b7549d2a478b34604"} Jul 12 22:14:36.744739 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:36.744674 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9qld6" podStartSLOduration=1.67651656 podStartE2EDuration="3.744659007s" totalImagesPullingTime="2.068142447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:14:36.743652538 +0000 UTC m=+258.612263360" watchObservedRunningTime="2026-07-12 22:14:36.744659007 +0000 UTC m=+258.613269829" Jul 12 22:14:36.938910 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:36.938879 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qxn4s"] Jul 12 22:14:36.958410 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:36.958362 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qxn4s"] Jul 12 22:14:36.958550 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:36.958469 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:36.961397 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:36.961353 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 12 22:14:36.961555 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:36.961433 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jj8rv\"" Jul 12 22:14:36.961714 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:36.961699 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 12 22:14:37.056801 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.056716 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8md22\" (UniqueName: \"kubernetes.io/projected/8cc7d6d9-def0-46a9-b0ef-017e61f9937f-kube-api-access-8md22\") pod \"cert-manager-webhook-6486ffd997-qxn4s\" (UID: \"8cc7d6d9-def0-46a9-b0ef-017e61f9937f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:37.056801 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.056750 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cc7d6d9-def0-46a9-b0ef-017e61f9937f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qxn4s\" (UID: \"8cc7d6d9-def0-46a9-b0ef-017e61f9937f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:37.157584 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.157543 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8md22\" (UniqueName: \"kubernetes.io/projected/8cc7d6d9-def0-46a9-b0ef-017e61f9937f-kube-api-access-8md22\") pod \"cert-manager-webhook-6486ffd997-qxn4s\" (UID: \"8cc7d6d9-def0-46a9-b0ef-017e61f9937f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:37.157584 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.157585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cc7d6d9-def0-46a9-b0ef-017e61f9937f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qxn4s\" (UID: \"8cc7d6d9-def0-46a9-b0ef-017e61f9937f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:37.179655 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.179631 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cc7d6d9-def0-46a9-b0ef-017e61f9937f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qxn4s\" (UID: \"8cc7d6d9-def0-46a9-b0ef-017e61f9937f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:37.179787 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.179641 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8md22\" (UniqueName: \"kubernetes.io/projected/8cc7d6d9-def0-46a9-b0ef-017e61f9937f-kube-api-access-8md22\") pod \"cert-manager-webhook-6486ffd997-qxn4s\" (UID: \"8cc7d6d9-def0-46a9-b0ef-017e61f9937f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:37.268030 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.267995 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:37.425116 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.425094 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qxn4s"] Jul 12 22:14:37.427467 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:14:37.427441 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cc7d6d9_def0_46a9_b0ef_017e61f9937f.slice/crio-6b1e48fa8373ca70b36b8e8e99539415ac5c5de0b08b22ad70673dd6c2cdb159 WatchSource:0}: Error finding container 6b1e48fa8373ca70b36b8e8e99539415ac5c5de0b08b22ad70673dd6c2cdb159: Status 404 returned error can't find the container with id 6b1e48fa8373ca70b36b8e8e99539415ac5c5de0b08b22ad70673dd6c2cdb159 Jul 12 22:14:37.722206 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.722178 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" event={"ID":"8cc7d6d9-def0-46a9-b0ef-017e61f9937f","Type":"ContainerStarted","Data":"6b1e48fa8373ca70b36b8e8e99539415ac5c5de0b08b22ad70673dd6c2cdb159"} Jul 12 22:14:37.760274 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.760243 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc"] Jul 12 22:14:37.795902 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.795876 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc"] Jul 12 22:14:37.796024 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.795978 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:37.800050 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.800026 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 22:14:37.800650 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.800634 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4prg8\"" Jul 12 22:14:37.800706 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.800634 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 22:14:37.863249 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.863221 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tm2n\" (UniqueName: \"kubernetes.io/projected/a3e1cb05-09b0-4116-b1ef-026baafa4843-kube-api-access-5tm2n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:37.863427 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.863263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:37.863427 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.863338 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:37.964617 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.964582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:37.964794 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.964639 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:37.964794 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.964685 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tm2n\" (UniqueName: \"kubernetes.io/projected/a3e1cb05-09b0-4116-b1ef-026baafa4843-kube-api-access-5tm2n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:37.965036 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.965015 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:37.965074 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.965026 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:37.973176 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:37.973126 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tm2n\" (UniqueName: \"kubernetes.io/projected/a3e1cb05-09b0-4116-b1ef-026baafa4843-kube-api-access-5tm2n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:38.105151 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:38.105118 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:38.257282 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:38.257259 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc"] Jul 12 22:14:38.258684 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:14:38.258658 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e1cb05_09b0_4116_b1ef_026baafa4843.slice/crio-6e3aa9e1777b371d8994fff1ef9bd434a27ca47a59939ba5a3424f061c0491f3 WatchSource:0}: Error finding container 6e3aa9e1777b371d8994fff1ef9bd434a27ca47a59939ba5a3424f061c0491f3: Status 404 returned error can't find the container with id 6e3aa9e1777b371d8994fff1ef9bd434a27ca47a59939ba5a3424f061c0491f3 Jul 12 22:14:38.727384 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:38.727346 2563 generic.go:356] "Generic (PLEG): container finished" podID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerID="52109a2b6d0b276de8bbcc25dc67c304212c8903594a23d6f71f437cbc2e4ce3" exitCode=0 Jul 12 22:14:38.727904 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:38.727432 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" event={"ID":"a3e1cb05-09b0-4116-b1ef-026baafa4843","Type":"ContainerDied","Data":"52109a2b6d0b276de8bbcc25dc67c304212c8903594a23d6f71f437cbc2e4ce3"} Jul 12 22:14:38.727904 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:38.727486 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" event={"ID":"a3e1cb05-09b0-4116-b1ef-026baafa4843","Type":"ContainerStarted","Data":"6e3aa9e1777b371d8994fff1ef9bd434a27ca47a59939ba5a3424f061c0491f3"} Jul 12 22:14:41.744182 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:41.744142 2563 generic.go:356] "Generic (PLEG): container finished" podID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerID="c5c47c57bfbd0a209c1c82ee01c4386aa67dbdf5103cf184dc1d1739d57e2190" exitCode=0 Jul 12 22:14:41.744636 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:41.744234 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" event={"ID":"a3e1cb05-09b0-4116-b1ef-026baafa4843","Type":"ContainerDied","Data":"c5c47c57bfbd0a209c1c82ee01c4386aa67dbdf5103cf184dc1d1739d57e2190"} Jul 12 22:14:41.745803 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:41.745777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" event={"ID":"8cc7d6d9-def0-46a9-b0ef-017e61f9937f","Type":"ContainerStarted","Data":"1f323e3b1f043728cda68479133f6a86af3d27825a37f202b2c18f84ad64aa89"} Jul 12 22:14:41.745899 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:41.745830 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:41.781098 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:41.781053 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" podStartSLOduration=2.240004538 podStartE2EDuration="5.781038007s" totalImagesPullingTime="3.541033469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:14:41.780111922 +0000 UTC m=+263.648722745" watchObservedRunningTime="2026-07-12 22:14:41.781038007 +0000 UTC m=+263.649648829" Jul 12 22:14:42.752123 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:42.752084 2563 generic.go:356] "Generic (PLEG): container finished" podID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerID="51027688f1398ceee1e2204393a4f16ad58a320e9e0a5e48e0462f5652b12269" exitCode=0 Jul 12 22:14:42.752569 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:42.752167 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" event={"ID":"a3e1cb05-09b0-4116-b1ef-026baafa4843","Type":"ContainerDied","Data":"51027688f1398ceee1e2204393a4f16ad58a320e9e0a5e48e0462f5652b12269"} Jul 12 22:14:43.875852 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:43.875829 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:44.011074 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.011003 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-bundle\") pod \"a3e1cb05-09b0-4116-b1ef-026baafa4843\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " Jul 12 22:14:44.011074 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.011051 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-util\") pod \"a3e1cb05-09b0-4116-b1ef-026baafa4843\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " Jul 12 22:14:44.011074 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.011075 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tm2n\" (UniqueName: \"kubernetes.io/projected/a3e1cb05-09b0-4116-b1ef-026baafa4843-kube-api-access-5tm2n\") pod \"a3e1cb05-09b0-4116-b1ef-026baafa4843\" (UID: \"a3e1cb05-09b0-4116-b1ef-026baafa4843\") " Jul 12 22:14:44.011478 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.011445 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-bundle" (OuterVolumeSpecName: "bundle") pod "a3e1cb05-09b0-4116-b1ef-026baafa4843" (UID: "a3e1cb05-09b0-4116-b1ef-026baafa4843"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:14:44.013248 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.013220 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e1cb05-09b0-4116-b1ef-026baafa4843-kube-api-access-5tm2n" (OuterVolumeSpecName: "kube-api-access-5tm2n") pod "a3e1cb05-09b0-4116-b1ef-026baafa4843" (UID: "a3e1cb05-09b0-4116-b1ef-026baafa4843"). InnerVolumeSpecName "kube-api-access-5tm2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:14:44.015449 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.015431 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-util" (OuterVolumeSpecName: "util") pod "a3e1cb05-09b0-4116-b1ef-026baafa4843" (UID: "a3e1cb05-09b0-4116-b1ef-026baafa4843"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:14:44.112469 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.112435 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tm2n\" (UniqueName: \"kubernetes.io/projected/a3e1cb05-09b0-4116-b1ef-026baafa4843-kube-api-access-5tm2n\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:14:44.112469 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.112463 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-bundle\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:14:44.112469 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.112475 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3e1cb05-09b0-4116-b1ef-026baafa4843-util\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:14:44.761279 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.761254 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" Jul 12 22:14:44.761439 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.761259 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwl2bc" event={"ID":"a3e1cb05-09b0-4116-b1ef-026baafa4843","Type":"ContainerDied","Data":"6e3aa9e1777b371d8994fff1ef9bd434a27ca47a59939ba5a3424f061c0491f3"} Jul 12 22:14:44.761439 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:44.761334 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e3aa9e1777b371d8994fff1ef9bd434a27ca47a59939ba5a3424f061c0491f3" Jul 12 22:14:47.754282 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.754244 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qxn4s" Jul 12 22:14:47.960913 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.960880 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rt2k6"] Jul 12 22:14:47.961166 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.961154 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerName="pull" Jul 12 22:14:47.961166 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.961167 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerName="pull" Jul 12 22:14:47.961248 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.961179 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerName="util" Jul 12 22:14:47.961248 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.961185 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerName="util" Jul 12 22:14:47.961248 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.961197 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerName="extract" Jul 12 22:14:47.961248 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.961203 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerName="extract" Jul 12 22:14:47.961248 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.961247 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e1cb05-09b0-4116-b1ef-026baafa4843" containerName="extract" Jul 12 22:14:47.963874 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.963857 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rt2k6" Jul 12 22:14:47.966547 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.966520 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nxsff\"" Jul 12 22:14:47.975617 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:47.975590 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rt2k6"] Jul 12 22:14:48.144572 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.144542 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmtms\" (UniqueName: \"kubernetes.io/projected/edda1bde-b39b-416e-a110-b312ad6ea3e3-kube-api-access-rmtms\") pod \"cert-manager-55b4d954d5-rt2k6\" (UID: \"edda1bde-b39b-416e-a110-b312ad6ea3e3\") " pod="cert-manager/cert-manager-55b4d954d5-rt2k6" Jul 12 22:14:48.144760 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.144589 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edda1bde-b39b-416e-a110-b312ad6ea3e3-bound-sa-token\") pod \"cert-manager-55b4d954d5-rt2k6\" (UID: \"edda1bde-b39b-416e-a110-b312ad6ea3e3\") " pod="cert-manager/cert-manager-55b4d954d5-rt2k6" Jul 12 22:14:48.245358 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.245318 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edda1bde-b39b-416e-a110-b312ad6ea3e3-bound-sa-token\") pod \"cert-manager-55b4d954d5-rt2k6\" (UID: \"edda1bde-b39b-416e-a110-b312ad6ea3e3\") " pod="cert-manager/cert-manager-55b4d954d5-rt2k6" Jul 12 22:14:48.245557 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.245422 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtms\" (UniqueName: \"kubernetes.io/projected/edda1bde-b39b-416e-a110-b312ad6ea3e3-kube-api-access-rmtms\") pod \"cert-manager-55b4d954d5-rt2k6\" (UID: \"edda1bde-b39b-416e-a110-b312ad6ea3e3\") " pod="cert-manager/cert-manager-55b4d954d5-rt2k6" Jul 12 22:14:48.255946 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.255912 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edda1bde-b39b-416e-a110-b312ad6ea3e3-bound-sa-token\") pod \"cert-manager-55b4d954d5-rt2k6\" (UID: \"edda1bde-b39b-416e-a110-b312ad6ea3e3\") " pod="cert-manager/cert-manager-55b4d954d5-rt2k6" Jul 12 22:14:48.256147 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.256127 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmtms\" (UniqueName: \"kubernetes.io/projected/edda1bde-b39b-416e-a110-b312ad6ea3e3-kube-api-access-rmtms\") pod \"cert-manager-55b4d954d5-rt2k6\" (UID: \"edda1bde-b39b-416e-a110-b312ad6ea3e3\") " pod="cert-manager/cert-manager-55b4d954d5-rt2k6" Jul 12 22:14:48.272069 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.272042 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rt2k6" Jul 12 22:14:48.401675 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.401608 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rt2k6"] Jul 12 22:14:48.404453 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:14:48.404423 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedda1bde_b39b_416e_a110_b312ad6ea3e3.slice/crio-c0e4dc491e3fb928dc7fa38ba4f06a028070f097a2639ebc43f0d5fd9934c8c0 WatchSource:0}: Error finding container c0e4dc491e3fb928dc7fa38ba4f06a028070f097a2639ebc43f0d5fd9934c8c0: Status 404 returned error can't find the container with id c0e4dc491e3fb928dc7fa38ba4f06a028070f097a2639ebc43f0d5fd9934c8c0 Jul 12 22:14:48.783342 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.783257 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rt2k6" event={"ID":"edda1bde-b39b-416e-a110-b312ad6ea3e3","Type":"ContainerStarted","Data":"0de8d394450709ff87c448451ab39e690d05f9cd71bb56cbe33e434ccf71d262"} Jul 12 22:14:48.783342 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:48.783303 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rt2k6" event={"ID":"edda1bde-b39b-416e-a110-b312ad6ea3e3","Type":"ContainerStarted","Data":"c0e4dc491e3fb928dc7fa38ba4f06a028070f097a2639ebc43f0d5fd9934c8c0"} Jul 12 22:14:56.748435 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.748361 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rt2k6" podStartSLOduration=9.74834553 podStartE2EDuration="9.74834553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:14:48.800091497 +0000 UTC m=+270.668702405" watchObservedRunningTime="2026-07-12 22:14:56.74834553 +0000 UTC m=+278.616956353" Jul 12 22:14:56.748808 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.748649 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj"] Jul 12 22:14:56.751991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.751975 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:56.755022 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.754999 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 22:14:56.755150 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.755039 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4prg8\"" Jul 12 22:14:56.755686 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.755672 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 22:14:56.762319 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.762298 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj"] Jul 12 22:14:56.911524 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.911492 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:56.911733 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.911551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49bxq\" (UniqueName: \"kubernetes.io/projected/10147d08-c47d-4e7d-baa2-f6bfb2429658-kube-api-access-49bxq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:56.911733 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:56.911609 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:57.012329 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.012238 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49bxq\" (UniqueName: \"kubernetes.io/projected/10147d08-c47d-4e7d-baa2-f6bfb2429658-kube-api-access-49bxq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:57.012329 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.012288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:57.012567 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.012360 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:57.012751 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.012731 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:57.012823 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.012772 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:57.021722 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.021697 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49bxq\" (UniqueName: \"kubernetes.io/projected/10147d08-c47d-4e7d-baa2-f6bfb2429658-kube-api-access-49bxq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:57.061918 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.061879 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:14:57.194067 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.194041 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj"] Jul 12 22:14:57.196228 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:14:57.196202 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10147d08_c47d_4e7d_baa2_f6bfb2429658.slice/crio-99251f36b2e4d3a0d84ef31fd9f7581052fe1f5cdd7f98f48e524cb2dc673618 WatchSource:0}: Error finding container 99251f36b2e4d3a0d84ef31fd9f7581052fe1f5cdd7f98f48e524cb2dc673618: Status 404 returned error can't find the container with id 99251f36b2e4d3a0d84ef31fd9f7581052fe1f5cdd7f98f48e524cb2dc673618 Jul 12 22:14:57.825644 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.825614 2563 generic.go:356] "Generic (PLEG): container finished" podID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerID="4e065682ad426093b6bed157152524c2f0b2b856d8af5325e63c415d515df86e" exitCode=0 Jul 12 22:14:57.826021 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.825668 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" event={"ID":"10147d08-c47d-4e7d-baa2-f6bfb2429658","Type":"ContainerDied","Data":"4e065682ad426093b6bed157152524c2f0b2b856d8af5325e63c415d515df86e"} Jul 12 22:14:57.826021 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:57.825704 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" event={"ID":"10147d08-c47d-4e7d-baa2-f6bfb2429658","Type":"ContainerStarted","Data":"99251f36b2e4d3a0d84ef31fd9f7581052fe1f5cdd7f98f48e524cb2dc673618"} Jul 12 22:14:58.831919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:58.831830 2563 generic.go:356] "Generic (PLEG): container finished" podID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerID="fce12869e8f05c8aa6f8764648ba74af97c57c770f916812aa0e5b7f970e9183" exitCode=0 Jul 12 22:14:58.832344 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:58.831910 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" event={"ID":"10147d08-c47d-4e7d-baa2-f6bfb2429658","Type":"ContainerDied","Data":"fce12869e8f05c8aa6f8764648ba74af97c57c770f916812aa0e5b7f970e9183"} Jul 12 22:14:59.838021 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:59.837987 2563 generic.go:356] "Generic (PLEG): container finished" podID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerID="2be725a5bb9cbe3ea3315160569c3c6695a7412f2be6f2cd66b92924fb349253" exitCode=0 Jul 12 22:14:59.838509 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:14:59.838086 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" event={"ID":"10147d08-c47d-4e7d-baa2-f6bfb2429658","Type":"ContainerDied","Data":"2be725a5bb9cbe3ea3315160569c3c6695a7412f2be6f2cd66b92924fb349253"} Jul 12 22:15:00.970311 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:00.970287 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:15:01.145295 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.145209 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49bxq\" (UniqueName: \"kubernetes.io/projected/10147d08-c47d-4e7d-baa2-f6bfb2429658-kube-api-access-49bxq\") pod \"10147d08-c47d-4e7d-baa2-f6bfb2429658\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " Jul 12 22:15:01.145466 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.145303 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-bundle\") pod \"10147d08-c47d-4e7d-baa2-f6bfb2429658\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " Jul 12 22:15:01.145466 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.145335 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-util\") pod \"10147d08-c47d-4e7d-baa2-f6bfb2429658\" (UID: \"10147d08-c47d-4e7d-baa2-f6bfb2429658\") " Jul 12 22:15:01.146031 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.146000 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-bundle" (OuterVolumeSpecName: "bundle") pod "10147d08-c47d-4e7d-baa2-f6bfb2429658" (UID: "10147d08-c47d-4e7d-baa2-f6bfb2429658"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:15:01.147423 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.147401 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10147d08-c47d-4e7d-baa2-f6bfb2429658-kube-api-access-49bxq" (OuterVolumeSpecName: "kube-api-access-49bxq") pod "10147d08-c47d-4e7d-baa2-f6bfb2429658" (UID: "10147d08-c47d-4e7d-baa2-f6bfb2429658"). InnerVolumeSpecName "kube-api-access-49bxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:15:01.151612 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.151568 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-util" (OuterVolumeSpecName: "util") pod "10147d08-c47d-4e7d-baa2-f6bfb2429658" (UID: "10147d08-c47d-4e7d-baa2-f6bfb2429658"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:15:01.246354 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.246316 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-bundle\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:15:01.246354 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.246347 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10147d08-c47d-4e7d-baa2-f6bfb2429658-util\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:15:01.246354 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.246360 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49bxq\" (UniqueName: \"kubernetes.io/projected/10147d08-c47d-4e7d-baa2-f6bfb2429658-kube-api-access-49bxq\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:15:01.850620 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.850582 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" event={"ID":"10147d08-c47d-4e7d-baa2-f6bfb2429658","Type":"ContainerDied","Data":"99251f36b2e4d3a0d84ef31fd9f7581052fe1f5cdd7f98f48e524cb2dc673618"} Jul 12 22:15:01.850620 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.850597 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affrnpj" Jul 12 22:15:01.850620 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:01.850628 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99251f36b2e4d3a0d84ef31fd9f7581052fe1f5cdd7f98f48e524cb2dc673618" Jul 12 22:15:09.162431 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.162395 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr"] Jul 12 22:15:09.162874 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.162684 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerName="pull" Jul 12 22:15:09.162874 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.162696 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerName="pull" Jul 12 22:15:09.162874 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.162707 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerName="extract" Jul 12 22:15:09.162874 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.162715 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerName="extract" Jul 12 22:15:09.162874 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.162733 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerName="util" Jul 12 22:15:09.162874 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.162738 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerName="util" Jul 12 22:15:09.162874 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.162783 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="10147d08-c47d-4e7d-baa2-f6bfb2429658" containerName="extract" Jul 12 22:15:09.167227 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.167068 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.172248 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.172225 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 22:15:09.172483 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.172464 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 22:15:09.173146 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.173128 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bq2zt\"" Jul 12 22:15:09.173401 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.173384 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 12 22:15:09.173459 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.173420 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 12 22:15:09.173459 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.173433 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 12 22:15:09.181882 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.181857 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr"] Jul 12 22:15:09.199722 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.199693 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/540c06e0-91a3-4174-ac0a-368ee41a8726-metrics-cert\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.199848 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.199724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/540c06e0-91a3-4174-ac0a-368ee41a8726-manager-config\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.199848 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.199752 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-792hr\" (UniqueName: \"kubernetes.io/projected/540c06e0-91a3-4174-ac0a-368ee41a8726-kube-api-access-792hr\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.199848 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.199805 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/540c06e0-91a3-4174-ac0a-368ee41a8726-cert\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.300467 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.300439 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/540c06e0-91a3-4174-ac0a-368ee41a8726-metrics-cert\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.300635 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.300480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/540c06e0-91a3-4174-ac0a-368ee41a8726-manager-config\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.300635 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.300515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-792hr\" (UniqueName: \"kubernetes.io/projected/540c06e0-91a3-4174-ac0a-368ee41a8726-kube-api-access-792hr\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.300635 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.300541 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/540c06e0-91a3-4174-ac0a-368ee41a8726-cert\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.301216 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.301190 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/540c06e0-91a3-4174-ac0a-368ee41a8726-manager-config\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.303076 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.303046 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/540c06e0-91a3-4174-ac0a-368ee41a8726-cert\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.303160 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.303131 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/540c06e0-91a3-4174-ac0a-368ee41a8726-metrics-cert\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.313586 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.313563 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-792hr\" (UniqueName: \"kubernetes.io/projected/540c06e0-91a3-4174-ac0a-368ee41a8726-kube-api-access-792hr\") pod \"lws-controller-manager-64b6b6c6c7-jt5zr\" (UID: \"540c06e0-91a3-4174-ac0a-368ee41a8726\") " pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.476780 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.476692 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:09.637857 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.637825 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr"] Jul 12 22:15:09.640309 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:15:09.640272 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod540c06e0_91a3_4174_ac0a_368ee41a8726.slice/crio-13baee037cab15f905066b2f07df8bcf5fc00c78f3769e9ad78f6b3e0070b983 WatchSource:0}: Error finding container 13baee037cab15f905066b2f07df8bcf5fc00c78f3769e9ad78f6b3e0070b983: Status 404 returned error can't find the container with id 13baee037cab15f905066b2f07df8bcf5fc00c78f3769e9ad78f6b3e0070b983 Jul 12 22:15:09.775929 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.775853 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj"] Jul 12 22:15:09.779981 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.779961 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.786646 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.786596 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 22:15:09.786916 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.786854 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 22:15:09.787357 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.787334 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 22:15:09.787513 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.787347 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 22:15:09.787586 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.787575 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-smkw7\"" Jul 12 22:15:09.804882 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.804831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33065002-dab1-48fc-9b08-a4eee34cd6b3-webhook-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.804882 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.804867 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33065002-dab1-48fc-9b08-a4eee34cd6b3-manifests\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.805076 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.804937 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33065002-dab1-48fc-9b08-a4eee34cd6b3-tmp\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.805076 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.805002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33065002-dab1-48fc-9b08-a4eee34cd6b3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.805076 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.805069 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28qpc\" (UniqueName: \"kubernetes.io/projected/33065002-dab1-48fc-9b08-a4eee34cd6b3-kube-api-access-28qpc\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.807293 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.807270 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj"] Jul 12 22:15:09.886060 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.886029 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" event={"ID":"540c06e0-91a3-4174-ac0a-368ee41a8726","Type":"ContainerStarted","Data":"13baee037cab15f905066b2f07df8bcf5fc00c78f3769e9ad78f6b3e0070b983"} Jul 12 22:15:09.905973 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.905948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33065002-dab1-48fc-9b08-a4eee34cd6b3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.906104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.905988 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28qpc\" (UniqueName: \"kubernetes.io/projected/33065002-dab1-48fc-9b08-a4eee34cd6b3-kube-api-access-28qpc\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.906104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.906011 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33065002-dab1-48fc-9b08-a4eee34cd6b3-webhook-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.906104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.906029 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33065002-dab1-48fc-9b08-a4eee34cd6b3-manifests\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.906104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.906048 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33065002-dab1-48fc-9b08-a4eee34cd6b3-tmp\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.906508 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.906484 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33065002-dab1-48fc-9b08-a4eee34cd6b3-manifests\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.908387 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.908350 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33065002-dab1-48fc-9b08-a4eee34cd6b3-tmp\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.908508 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.908488 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33065002-dab1-48fc-9b08-a4eee34cd6b3-webhook-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.908563 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.908503 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33065002-dab1-48fc-9b08-a4eee34cd6b3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:09.914980 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:09.914961 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28qpc\" (UniqueName: \"kubernetes.io/projected/33065002-dab1-48fc-9b08-a4eee34cd6b3-kube-api-access-28qpc\") pod \"opendatahub-operator-controller-manager-56d7986bbd-47jnj\" (UID: \"33065002-dab1-48fc-9b08-a4eee34cd6b3\") " pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:10.090552 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:10.090523 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:10.246017 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:10.245982 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj"] Jul 12 22:15:10.254519 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:15:10.254493 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33065002_dab1_48fc_9b08_a4eee34cd6b3.slice/crio-09ecb4979cefba38124c754fc699d678c049baf86fa71263a878c584b2453dc4 WatchSource:0}: Error finding container 09ecb4979cefba38124c754fc699d678c049baf86fa71263a878c584b2453dc4: Status 404 returned error can't find the container with id 09ecb4979cefba38124c754fc699d678c049baf86fa71263a878c584b2453dc4 Jul 12 22:15:10.892792 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:10.892749 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" event={"ID":"33065002-dab1-48fc-9b08-a4eee34cd6b3","Type":"ContainerStarted","Data":"09ecb4979cefba38124c754fc699d678c049baf86fa71263a878c584b2453dc4"} Jul 12 22:15:13.912127 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:13.912095 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" event={"ID":"540c06e0-91a3-4174-ac0a-368ee41a8726","Type":"ContainerStarted","Data":"77cbcded207fe539906d57aab8674cb3d26ae592b157cf1e2ed16abc5dce6671"} Jul 12 22:15:13.912567 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:13.912172 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:13.913591 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:13.913570 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" event={"ID":"33065002-dab1-48fc-9b08-a4eee34cd6b3","Type":"ContainerStarted","Data":"d2b06a089f282d1d55df41ae9cc687d1602d03df64907f3824815f6bae4c0bb9"} Jul 12 22:15:13.937050 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:13.936998 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" podStartSLOduration=1.169043838 podStartE2EDuration="4.936982325s" totalImagesPullingTime="3.767938487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:15:13.934896462 +0000 UTC m=+295.803507287" watchObservedRunningTime="2026-07-12 22:15:13.936982325 +0000 UTC m=+295.805593220" Jul 12 22:15:14.919561 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:14.919525 2563 generic.go:356] "Generic (PLEG): container finished" podID="33065002-dab1-48fc-9b08-a4eee34cd6b3" containerID="d2b06a089f282d1d55df41ae9cc687d1602d03df64907f3824815f6bae4c0bb9" exitCode=0 Jul 12 22:15:14.919973 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:14.919619 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" event={"ID":"33065002-dab1-48fc-9b08-a4eee34cd6b3","Type":"ContainerDied","Data":"d2b06a089f282d1d55df41ae9cc687d1602d03df64907f3824815f6bae4c0bb9"} Jul 12 22:15:15.926911 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:15.926875 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" event={"ID":"33065002-dab1-48fc-9b08-a4eee34cd6b3","Type":"ContainerStarted","Data":"de84b20a13bde727dee91bcac8fc58dd960868ef44c749b63aac41d9eed8189b"} Jul 12 22:15:15.927312 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:15.926988 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:15.954754 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:15.954695 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" podStartSLOduration=2.375834098 podStartE2EDuration="6.954673556s" totalImagesPullingTime="3.55105291s" totalInitContainerRuntime="1.027786548s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:15:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:15:15.954087929 +0000 UTC m=+297.822698754" watchObservedRunningTime="2026-07-12 22:15:15.954673556 +0000 UTC m=+297.823284379" Jul 12 22:15:18.562245 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:18.562210 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:15:18.565911 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:18.565885 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:15:18.570470 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:18.570443 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:15:18.574155 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:18.574131 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:15:18.574319 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:18.574304 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 22:15:24.922346 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:24.922312 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64b6b6c6c7-jt5zr" Jul 12 22:15:26.933886 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:26.933857 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56d7986bbd-47jnj" Jul 12 22:15:29.432757 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.432729 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5747db99d8-78dk4"] Jul 12 22:15:29.435857 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.435840 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.439755 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.439732 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 12 22:15:29.440042 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.440025 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 12 22:15:29.440350 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.440335 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 22:15:29.440664 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.440648 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 22:15:29.441171 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.441153 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-26nhb\"" Jul 12 22:15:29.455250 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.455224 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5747db99d8-78dk4"] Jul 12 22:15:29.563334 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.563301 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/938ac42e-556c-483d-9306-cd0944e21be2-tls-certs\") pod \"kube-auth-proxy-5747db99d8-78dk4\" (UID: \"938ac42e-556c-483d-9306-cd0944e21be2\") " pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.563525 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.563398 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfvhx\" (UniqueName: \"kubernetes.io/projected/938ac42e-556c-483d-9306-cd0944e21be2-kube-api-access-kfvhx\") pod \"kube-auth-proxy-5747db99d8-78dk4\" (UID: \"938ac42e-556c-483d-9306-cd0944e21be2\") " pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.563525 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.563441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/938ac42e-556c-483d-9306-cd0944e21be2-tmp\") pod \"kube-auth-proxy-5747db99d8-78dk4\" (UID: \"938ac42e-556c-483d-9306-cd0944e21be2\") " pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.664734 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.664702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/938ac42e-556c-483d-9306-cd0944e21be2-tls-certs\") pod \"kube-auth-proxy-5747db99d8-78dk4\" (UID: \"938ac42e-556c-483d-9306-cd0944e21be2\") " pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.664906 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.664755 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfvhx\" (UniqueName: \"kubernetes.io/projected/938ac42e-556c-483d-9306-cd0944e21be2-kube-api-access-kfvhx\") pod \"kube-auth-proxy-5747db99d8-78dk4\" (UID: \"938ac42e-556c-483d-9306-cd0944e21be2\") " pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.664906 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.664786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/938ac42e-556c-483d-9306-cd0944e21be2-tmp\") pod \"kube-auth-proxy-5747db99d8-78dk4\" (UID: \"938ac42e-556c-483d-9306-cd0944e21be2\") " pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.667032 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.667010 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/938ac42e-556c-483d-9306-cd0944e21be2-tmp\") pod \"kube-auth-proxy-5747db99d8-78dk4\" (UID: \"938ac42e-556c-483d-9306-cd0944e21be2\") " pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.667239 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.667220 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/938ac42e-556c-483d-9306-cd0944e21be2-tls-certs\") pod \"kube-auth-proxy-5747db99d8-78dk4\" (UID: \"938ac42e-556c-483d-9306-cd0944e21be2\") " pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.703817 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.703747 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfvhx\" (UniqueName: \"kubernetes.io/projected/938ac42e-556c-483d-9306-cd0944e21be2-kube-api-access-kfvhx\") pod \"kube-auth-proxy-5747db99d8-78dk4\" (UID: \"938ac42e-556c-483d-9306-cd0944e21be2\") " pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.744989 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.744960 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" Jul 12 22:15:29.907630 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.907609 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5747db99d8-78dk4"] Jul 12 22:15:29.910129 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:15:29.910103 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod938ac42e_556c_483d_9306_cd0944e21be2.slice/crio-7715dcdff006a8c28e77d214037b042a0a3458f9aac78d9106e930f44abba612 WatchSource:0}: Error finding container 7715dcdff006a8c28e77d214037b042a0a3458f9aac78d9106e930f44abba612: Status 404 returned error can't find the container with id 7715dcdff006a8c28e77d214037b042a0a3458f9aac78d9106e930f44abba612 Jul 12 22:15:29.911938 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.911914 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:15:29.993287 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:29.993189 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" event={"ID":"938ac42e-556c-483d-9306-cd0944e21be2","Type":"ContainerStarted","Data":"7715dcdff006a8c28e77d214037b042a0a3458f9aac78d9106e930f44abba612"} Jul 12 22:15:33.008032 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:33.007997 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" event={"ID":"938ac42e-556c-483d-9306-cd0944e21be2","Type":"ContainerStarted","Data":"f9cd7000e1c5304d7b4ec35314fff6baf6b8a14e9c9956fb5d3ba1026dffe811"} Jul 12 22:15:33.034478 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:15:33.034434 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5747db99d8-78dk4" podStartSLOduration=1.6838202820000001 podStartE2EDuration="4.034420034s" totalImagesPullingTime="2.350599752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:15:33.032581905 +0000 UTC m=+314.901192729" watchObservedRunningTime="2026-07-12 22:15:33.034420034 +0000 UTC m=+314.903030858" Jul 12 22:16:45.398806 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.398720 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg"] Jul 12 22:16:45.401897 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.401882 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.404549 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.404518 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 22:16:45.405304 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.405285 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 22:16:45.405425 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.405291 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qs8cq\"" Jul 12 22:16:45.409772 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.409746 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg"] Jul 12 22:16:45.466304 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.466274 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.466304 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.466308 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.466565 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.466345 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljl89\" (UniqueName: \"kubernetes.io/projected/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-kube-api-access-ljl89\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.567432 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.567401 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.567432 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.567433 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.567697 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.567469 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljl89\" (UniqueName: \"kubernetes.io/projected/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-kube-api-access-ljl89\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.567904 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.567881 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.567984 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.567908 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.575696 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.575674 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljl89\" (UniqueName: \"kubernetes.io/projected/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-kube-api-access-ljl89\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.711955 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.711877 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:45.846516 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:45.846493 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg"] Jul 12 22:16:45.848084 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:16:45.848052 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8fe40c3_1c50_48bb_b383_ceecc64a0ad7.slice/crio-0dc37d57713d4b2a11a52fa5a9a35d3ecd5f157575d700447a51883244d172d2 WatchSource:0}: Error finding container 0dc37d57713d4b2a11a52fa5a9a35d3ecd5f157575d700447a51883244d172d2: Status 404 returned error can't find the container with id 0dc37d57713d4b2a11a52fa5a9a35d3ecd5f157575d700447a51883244d172d2 Jul 12 22:16:46.194784 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.194751 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749"] Jul 12 22:16:46.197959 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.197943 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.206003 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.205979 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749"] Jul 12 22:16:46.273978 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.273946 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.274100 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.273998 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.274100 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.274073 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kwpl\" (UniqueName: \"kubernetes.io/projected/3e305a11-031b-4458-a56c-ee28f33337cf-kube-api-access-6kwpl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.325050 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.325018 2563 generic.go:356] "Generic (PLEG): container finished" podID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerID="d656635d02e4335d973d859ca4926543a774af37d36530958e5f12df2039146d" exitCode=0 Jul 12 22:16:46.325191 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.325106 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" event={"ID":"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7","Type":"ContainerDied","Data":"d656635d02e4335d973d859ca4926543a774af37d36530958e5f12df2039146d"} Jul 12 22:16:46.325191 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.325153 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" event={"ID":"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7","Type":"ContainerStarted","Data":"0dc37d57713d4b2a11a52fa5a9a35d3ecd5f157575d700447a51883244d172d2"} Jul 12 22:16:46.374879 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.374848 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.375042 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.375019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.375142 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.375124 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kwpl\" (UniqueName: \"kubernetes.io/projected/3e305a11-031b-4458-a56c-ee28f33337cf-kube-api-access-6kwpl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.375297 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.375279 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.375355 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.375302 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.387276 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.387253 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kwpl\" (UniqueName: \"kubernetes.io/projected/3e305a11-031b-4458-a56c-ee28f33337cf-kube-api-access-6kwpl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.507525 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.507456 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:46.638867 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.638845 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749"] Jul 12 22:16:46.641143 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:16:46.641112 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e305a11_031b_4458_a56c_ee28f33337cf.slice/crio-f468eeb056558d44721cec64537b5583f3b80ebb3f1f44c1cdf3e2937dd0c7cc WatchSource:0}: Error finding container f468eeb056558d44721cec64537b5583f3b80ebb3f1f44c1cdf3e2937dd0c7cc: Status 404 returned error can't find the container with id f468eeb056558d44721cec64537b5583f3b80ebb3f1f44c1cdf3e2937dd0c7cc Jul 12 22:16:46.995306 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.995262 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn"] Jul 12 22:16:46.999702 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:46.999660 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.010218 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.010192 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn"] Jul 12 22:16:47.082243 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.082205 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.082415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.082263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czjw8\" (UniqueName: \"kubernetes.io/projected/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-kube-api-access-czjw8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.082415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.082342 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.183552 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.183521 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjw8\" (UniqueName: \"kubernetes.io/projected/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-kube-api-access-czjw8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.183680 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.183578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.183680 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.183615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.183969 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.183949 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.184025 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.183965 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.192553 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.192530 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czjw8\" (UniqueName: \"kubernetes.io/projected/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-kube-api-access-czjw8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.313239 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.313162 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:47.331823 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.331791 2563 generic.go:356] "Generic (PLEG): container finished" podID="3e305a11-031b-4458-a56c-ee28f33337cf" containerID="72233de967be6616fec964af97c658f2fef95974a48012957ab083894c22f509" exitCode=0 Jul 12 22:16:47.331929 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.331874 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" event={"ID":"3e305a11-031b-4458-a56c-ee28f33337cf","Type":"ContainerDied","Data":"72233de967be6616fec964af97c658f2fef95974a48012957ab083894c22f509"} Jul 12 22:16:47.331929 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.331920 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" event={"ID":"3e305a11-031b-4458-a56c-ee28f33337cf","Type":"ContainerStarted","Data":"f468eeb056558d44721cec64537b5583f3b80ebb3f1f44c1cdf3e2937dd0c7cc"} Jul 12 22:16:47.334057 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.334036 2563 generic.go:356] "Generic (PLEG): container finished" podID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerID="fbac5956fe05a629d56fce4b8a8e166f7bd78c8a04104043520c00542ad47636" exitCode=0 Jul 12 22:16:47.334131 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.334094 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" event={"ID":"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7","Type":"ContainerDied","Data":"fbac5956fe05a629d56fce4b8a8e166f7bd78c8a04104043520c00542ad47636"} Jul 12 22:16:47.450573 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.450530 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn"] Jul 12 22:16:47.452644 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:16:47.452618 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20e11d5_636c_4fe6_a3a8_6a0f0eb610b9.slice/crio-862781462bf7d1feae5d90bd06d8714237c0715fb6b70a12a02fea22cb514fcb WatchSource:0}: Error finding container 862781462bf7d1feae5d90bd06d8714237c0715fb6b70a12a02fea22cb514fcb: Status 404 returned error can't find the container with id 862781462bf7d1feae5d90bd06d8714237c0715fb6b70a12a02fea22cb514fcb Jul 12 22:16:47.801953 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.801925 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb"] Jul 12 22:16:47.805153 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.805136 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:47.815423 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.815397 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb"] Jul 12 22:16:47.888445 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.888419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:47.888592 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.888459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:47.888592 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.888483 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shw4p\" (UniqueName: \"kubernetes.io/projected/d2020f75-296f-42a0-b222-942007ead610-kube-api-access-shw4p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:47.989290 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.989251 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:47.989492 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.989305 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:47.989492 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.989331 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shw4p\" (UniqueName: \"kubernetes.io/projected/d2020f75-296f-42a0-b222-942007ead610-kube-api-access-shw4p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:47.989699 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.989677 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:47.989804 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.989784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:47.999948 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:47.999918 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shw4p\" (UniqueName: \"kubernetes.io/projected/d2020f75-296f-42a0-b222-942007ead610-kube-api-access-shw4p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:48.114092 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.114069 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:48.253900 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.253876 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb"] Jul 12 22:16:48.256313 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:16:48.256286 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2020f75_296f_42a0_b222_942007ead610.slice/crio-7da7fea084318512c3c3a65943d5018347c5371f2cbf0b35a8e1c0fd45bae889 WatchSource:0}: Error finding container 7da7fea084318512c3c3a65943d5018347c5371f2cbf0b35a8e1c0fd45bae889: Status 404 returned error can't find the container with id 7da7fea084318512c3c3a65943d5018347c5371f2cbf0b35a8e1c0fd45bae889 Jul 12 22:16:48.341631 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.341602 2563 generic.go:356] "Generic (PLEG): container finished" podID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerID="c12a01a4198c310545c3ed7e743d5d36b436ab368ab42797acf15998eebde4f5" exitCode=0 Jul 12 22:16:48.341800 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.341695 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" event={"ID":"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7","Type":"ContainerDied","Data":"c12a01a4198c310545c3ed7e743d5d36b436ab368ab42797acf15998eebde4f5"} Jul 12 22:16:48.343215 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.343182 2563 generic.go:356] "Generic (PLEG): container finished" podID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerID="9736bd648a817098527263bdcc232405615f1ad3b6d068291bbfc25d31866e86" exitCode=0 Jul 12 22:16:48.343351 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.343266 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" event={"ID":"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9","Type":"ContainerDied","Data":"9736bd648a817098527263bdcc232405615f1ad3b6d068291bbfc25d31866e86"} Jul 12 22:16:48.343351 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.343312 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" event={"ID":"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9","Type":"ContainerStarted","Data":"862781462bf7d1feae5d90bd06d8714237c0715fb6b70a12a02fea22cb514fcb"} Jul 12 22:16:48.345133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.345109 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" event={"ID":"d2020f75-296f-42a0-b222-942007ead610","Type":"ContainerStarted","Data":"be59b185d2ec0bed4311c0940f3a35a3035bfc3750c026c8e2b75352ed24f017"} Jul 12 22:16:48.345243 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.345155 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" event={"ID":"d2020f75-296f-42a0-b222-942007ead610","Type":"ContainerStarted","Data":"7da7fea084318512c3c3a65943d5018347c5371f2cbf0b35a8e1c0fd45bae889"} Jul 12 22:16:48.347137 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.347115 2563 generic.go:356] "Generic (PLEG): container finished" podID="3e305a11-031b-4458-a56c-ee28f33337cf" containerID="9f6d0b8ba63462ff69c07b05f84d468c493097f2ab798a78c67650ba86917ad2" exitCode=0 Jul 12 22:16:48.347225 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:48.347198 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" event={"ID":"3e305a11-031b-4458-a56c-ee28f33337cf","Type":"ContainerDied","Data":"9f6d0b8ba63462ff69c07b05f84d468c493097f2ab798a78c67650ba86917ad2"} Jul 12 22:16:49.354158 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.354121 2563 generic.go:356] "Generic (PLEG): container finished" podID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerID="5f8651208efc3ee9d4dcfa1b3642b5e96d140450c5a5e20979a91b9ce4cdde65" exitCode=0 Jul 12 22:16:49.354561 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.354204 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" event={"ID":"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9","Type":"ContainerDied","Data":"5f8651208efc3ee9d4dcfa1b3642b5e96d140450c5a5e20979a91b9ce4cdde65"} Jul 12 22:16:49.355761 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.355742 2563 generic.go:356] "Generic (PLEG): container finished" podID="d2020f75-296f-42a0-b222-942007ead610" containerID="be59b185d2ec0bed4311c0940f3a35a3035bfc3750c026c8e2b75352ed24f017" exitCode=0 Jul 12 22:16:49.355838 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.355819 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" event={"ID":"d2020f75-296f-42a0-b222-942007ead610","Type":"ContainerDied","Data":"be59b185d2ec0bed4311c0940f3a35a3035bfc3750c026c8e2b75352ed24f017"} Jul 12 22:16:49.357896 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.357870 2563 generic.go:356] "Generic (PLEG): container finished" podID="3e305a11-031b-4458-a56c-ee28f33337cf" containerID="5cf002b97913b7a66f5a02df4204456e111b3e50a3e83628dda3ddca5e0247f6" exitCode=0 Jul 12 22:16:49.357989 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.357953 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" event={"ID":"3e305a11-031b-4458-a56c-ee28f33337cf","Type":"ContainerDied","Data":"5cf002b97913b7a66f5a02df4204456e111b3e50a3e83628dda3ddca5e0247f6"} Jul 12 22:16:49.494720 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.494693 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:49.543652 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:16:49.543613 2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20e11d5_636c_4fe6_a3a8_6a0f0eb610b9.slice/crio-b06ca8907df7ed819ff8de3eeb2e4bcb26ea5daa041968fba570363c29710f69.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20e11d5_636c_4fe6_a3a8_6a0f0eb610b9.slice/crio-conmon-b06ca8907df7ed819ff8de3eeb2e4bcb26ea5daa041968fba570363c29710f69.scope\": RecentStats: unable to find data in memory cache]" Jul 12 22:16:49.600790 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.600745 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-util\") pod \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " Jul 12 22:16:49.600981 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.600823 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljl89\" (UniqueName: \"kubernetes.io/projected/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-kube-api-access-ljl89\") pod \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " Jul 12 22:16:49.600981 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.600868 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-bundle\") pod \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\" (UID: \"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7\") " Jul 12 22:16:49.601435 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.601402 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-bundle" (OuterVolumeSpecName: "bundle") pod "a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" (UID: "a8fe40c3-1c50-48bb-b383-ceecc64a0ad7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:16:49.603226 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.603198 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-kube-api-access-ljl89" (OuterVolumeSpecName: "kube-api-access-ljl89") pod "a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" (UID: "a8fe40c3-1c50-48bb-b383-ceecc64a0ad7"). InnerVolumeSpecName "kube-api-access-ljl89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:16:49.606484 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.606429 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-util" (OuterVolumeSpecName: "util") pod "a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" (UID: "a8fe40c3-1c50-48bb-b383-ceecc64a0ad7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:16:49.701528 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.701475 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-bundle\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:49.701528 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.701515 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-util\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:49.701528 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:49.701526 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljl89\" (UniqueName: \"kubernetes.io/projected/a8fe40c3-1c50-48bb-b383-ceecc64a0ad7-kube-api-access-ljl89\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:50.364493 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.364459 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" event={"ID":"a8fe40c3-1c50-48bb-b383-ceecc64a0ad7","Type":"ContainerDied","Data":"0dc37d57713d4b2a11a52fa5a9a35d3ecd5f157575d700447a51883244d172d2"} Jul 12 22:16:50.364898 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.364502 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg" Jul 12 22:16:50.364898 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.364506 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dc37d57713d4b2a11a52fa5a9a35d3ecd5f157575d700447a51883244d172d2" Jul 12 22:16:50.366569 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.366544 2563 generic.go:356] "Generic (PLEG): container finished" podID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerID="b06ca8907df7ed819ff8de3eeb2e4bcb26ea5daa041968fba570363c29710f69" exitCode=0 Jul 12 22:16:50.366683 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.366628 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" event={"ID":"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9","Type":"ContainerDied","Data":"b06ca8907df7ed819ff8de3eeb2e4bcb26ea5daa041968fba570363c29710f69"} Jul 12 22:16:50.368393 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.368352 2563 generic.go:356] "Generic (PLEG): container finished" podID="d2020f75-296f-42a0-b222-942007ead610" containerID="0f28bb88dc018bbd2c7749a4312f2b0a47d96ac41fb8d30abcd4837a59d8524e" exitCode=0 Jul 12 22:16:50.368502 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.368398 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" event={"ID":"d2020f75-296f-42a0-b222-942007ead610","Type":"ContainerDied","Data":"0f28bb88dc018bbd2c7749a4312f2b0a47d96ac41fb8d30abcd4837a59d8524e"} Jul 12 22:16:50.492852 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.492828 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:50.609601 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.609573 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-util\") pod \"3e305a11-031b-4458-a56c-ee28f33337cf\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " Jul 12 22:16:50.609725 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.609647 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-bundle\") pod \"3e305a11-031b-4458-a56c-ee28f33337cf\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " Jul 12 22:16:50.609725 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.609689 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kwpl\" (UniqueName: \"kubernetes.io/projected/3e305a11-031b-4458-a56c-ee28f33337cf-kube-api-access-6kwpl\") pod \"3e305a11-031b-4458-a56c-ee28f33337cf\" (UID: \"3e305a11-031b-4458-a56c-ee28f33337cf\") " Jul 12 22:16:50.610087 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.610064 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-bundle" (OuterVolumeSpecName: "bundle") pod "3e305a11-031b-4458-a56c-ee28f33337cf" (UID: "3e305a11-031b-4458-a56c-ee28f33337cf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:16:50.611862 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.611841 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e305a11-031b-4458-a56c-ee28f33337cf-kube-api-access-6kwpl" (OuterVolumeSpecName: "kube-api-access-6kwpl") pod "3e305a11-031b-4458-a56c-ee28f33337cf" (UID: "3e305a11-031b-4458-a56c-ee28f33337cf"). InnerVolumeSpecName "kube-api-access-6kwpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:16:50.614941 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.614896 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-util" (OuterVolumeSpecName: "util") pod "3e305a11-031b-4458-a56c-ee28f33337cf" (UID: "3e305a11-031b-4458-a56c-ee28f33337cf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:16:50.710661 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.710629 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-bundle\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:50.710661 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.710659 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kwpl\" (UniqueName: \"kubernetes.io/projected/3e305a11-031b-4458-a56c-ee28f33337cf-kube-api-access-6kwpl\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:50.710886 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:50.710673 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e305a11-031b-4458-a56c-ee28f33337cf-util\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:51.375254 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.375218 2563 generic.go:356] "Generic (PLEG): container finished" podID="d2020f75-296f-42a0-b222-942007ead610" containerID="86deb9353b800191b171c1558e66ecccc0c2618a2da110da3868cbd9cc7b502f" exitCode=0 Jul 12 22:16:51.375685 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.375265 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" event={"ID":"d2020f75-296f-42a0-b222-942007ead610","Type":"ContainerDied","Data":"86deb9353b800191b171c1558e66ecccc0c2618a2da110da3868cbd9cc7b502f"} Jul 12 22:16:51.377185 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.377165 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" Jul 12 22:16:51.377291 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.377165 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749" event={"ID":"3e305a11-031b-4458-a56c-ee28f33337cf","Type":"ContainerDied","Data":"f468eeb056558d44721cec64537b5583f3b80ebb3f1f44c1cdf3e2937dd0c7cc"} Jul 12 22:16:51.377291 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.377221 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f468eeb056558d44721cec64537b5583f3b80ebb3f1f44c1cdf3e2937dd0c7cc" Jul 12 22:16:51.502667 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.502639 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:51.618285 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.618251 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-util\") pod \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " Jul 12 22:16:51.618472 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.618315 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-bundle\") pod \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " Jul 12 22:16:51.618472 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.618345 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czjw8\" (UniqueName: \"kubernetes.io/projected/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-kube-api-access-czjw8\") pod \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\" (UID: \"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9\") " Jul 12 22:16:51.618978 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.618954 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-bundle" (OuterVolumeSpecName: "bundle") pod "b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" (UID: "b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:16:51.620649 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.620625 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-kube-api-access-czjw8" (OuterVolumeSpecName: "kube-api-access-czjw8") pod "b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" (UID: "b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9"). InnerVolumeSpecName "kube-api-access-czjw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:16:51.623459 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.623437 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-util" (OuterVolumeSpecName: "util") pod "b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" (UID: "b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:16:51.719183 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.719091 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-bundle\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:51.719183 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.719125 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czjw8\" (UniqueName: \"kubernetes.io/projected/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-kube-api-access-czjw8\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:51.719183 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:51.719138 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9-util\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:52.383993 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.383959 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" Jul 12 22:16:52.383993 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.383987 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn" event={"ID":"b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9","Type":"ContainerDied","Data":"862781462bf7d1feae5d90bd06d8714237c0715fb6b70a12a02fea22cb514fcb"} Jul 12 22:16:52.384450 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.384036 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="862781462bf7d1feae5d90bd06d8714237c0715fb6b70a12a02fea22cb514fcb" Jul 12 22:16:52.514635 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.514610 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:52.627647 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.627619 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-util\") pod \"d2020f75-296f-42a0-b222-942007ead610\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " Jul 12 22:16:52.627847 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.627678 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-bundle\") pod \"d2020f75-296f-42a0-b222-942007ead610\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " Jul 12 22:16:52.627847 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.627717 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shw4p\" (UniqueName: \"kubernetes.io/projected/d2020f75-296f-42a0-b222-942007ead610-kube-api-access-shw4p\") pod \"d2020f75-296f-42a0-b222-942007ead610\" (UID: \"d2020f75-296f-42a0-b222-942007ead610\") " Jul 12 22:16:52.628205 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.628179 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-bundle" (OuterVolumeSpecName: "bundle") pod "d2020f75-296f-42a0-b222-942007ead610" (UID: "d2020f75-296f-42a0-b222-942007ead610"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:16:52.630235 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.630210 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2020f75-296f-42a0-b222-942007ead610-kube-api-access-shw4p" (OuterVolumeSpecName: "kube-api-access-shw4p") pod "d2020f75-296f-42a0-b222-942007ead610" (UID: "d2020f75-296f-42a0-b222-942007ead610"). InnerVolumeSpecName "kube-api-access-shw4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:16:52.633404 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.633383 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-util" (OuterVolumeSpecName: "util") pod "d2020f75-296f-42a0-b222-942007ead610" (UID: "d2020f75-296f-42a0-b222-942007ead610"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:16:52.728718 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.728689 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shw4p\" (UniqueName: \"kubernetes.io/projected/d2020f75-296f-42a0-b222-942007ead610-kube-api-access-shw4p\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:52.728718 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.728718 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-util\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:52.728870 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:52.728733 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2020f75-296f-42a0-b222-942007ead610-bundle\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:16:53.390351 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:53.390314 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" event={"ID":"d2020f75-296f-42a0-b222-942007ead610","Type":"ContainerDied","Data":"7da7fea084318512c3c3a65943d5018347c5371f2cbf0b35a8e1c0fd45bae889"} Jul 12 22:16:53.390743 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:53.390357 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb" Jul 12 22:16:53.390743 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:16:53.390364 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7da7fea084318512c3c3a65943d5018347c5371f2cbf0b35a8e1c0fd45bae889" Jul 12 22:17:30.604732 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.604691 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g"] Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605027 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerName="extract" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605041 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerName="extract" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605048 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2020f75-296f-42a0-b222-942007ead610" containerName="pull" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605053 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2020f75-296f-42a0-b222-942007ead610" containerName="pull" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605061 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerName="util" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605066 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerName="util" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605072 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerName="util" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605077 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerName="util" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605084 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2020f75-296f-42a0-b222-942007ead610" containerName="util" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605090 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2020f75-296f-42a0-b222-942007ead610" containerName="util" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605103 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerName="extract" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605111 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerName="extract" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605124 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e305a11-031b-4458-a56c-ee28f33337cf" containerName="util" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605129 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e305a11-031b-4458-a56c-ee28f33337cf" containerName="util" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605135 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerName="pull" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605139 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerName="pull" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605147 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e305a11-031b-4458-a56c-ee28f33337cf" containerName="pull" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605151 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e305a11-031b-4458-a56c-ee28f33337cf" containerName="pull" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605157 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e305a11-031b-4458-a56c-ee28f33337cf" containerName="extract" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605161 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e305a11-031b-4458-a56c-ee28f33337cf" containerName="extract" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605167 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerName="pull" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605171 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerName="pull" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605179 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2020f75-296f-42a0-b222-942007ead610" containerName="extract" Jul 12 22:17:30.605193 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605186 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2020f75-296f-42a0-b222-942007ead610" containerName="extract" Jul 12 22:17:30.605890 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605233 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8fe40c3-1c50-48bb-b383-ceecc64a0ad7" containerName="extract" Jul 12 22:17:30.605890 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605242 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2020f75-296f-42a0-b222-942007ead610" containerName="extract" Jul 12 22:17:30.605890 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605248 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e305a11-031b-4458-a56c-ee28f33337cf" containerName="extract" Jul 12 22:17:30.605890 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.605255 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9" containerName="extract" Jul 12 22:17:30.609516 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.609499 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:30.612901 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.612881 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 22:17:30.613057 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.613025 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 22:17:30.613135 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.613116 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8jsv5\"" Jul 12 22:17:30.620886 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.620863 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g"] Jul 12 22:17:30.629907 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.629887 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jck5g\" (UID: \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:30.630036 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.629947 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzds5\" (UniqueName: \"kubernetes.io/projected/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-kube-api-access-fzds5\") pod \"kuadrant-operator-controller-manager-55467d9f7-jck5g\" (UID: \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:30.731270 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.731232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzds5\" (UniqueName: \"kubernetes.io/projected/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-kube-api-access-fzds5\") pod \"kuadrant-operator-controller-manager-55467d9f7-jck5g\" (UID: \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:30.731471 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.731289 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jck5g\" (UID: \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:30.731662 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.731645 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jck5g\" (UID: \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:30.739812 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.739784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzds5\" (UniqueName: \"kubernetes.io/projected/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-kube-api-access-fzds5\") pod \"kuadrant-operator-controller-manager-55467d9f7-jck5g\" (UID: \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:30.871565 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.871475 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g"] Jul 12 22:17:30.871733 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.871722 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:30.877922 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.877895 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g"] Jul 12 22:17:30.898997 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.898969 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2"] Jul 12 22:17:30.902202 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.902181 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:30.928138 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.928109 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2"] Jul 12 22:17:30.933150 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.932982 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ba531e-0810-4735-9466-d3410536a132-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-q4mm2\" (UID: \"71ba531e-0810-4735-9466-d3410536a132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:30.933150 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:30.933082 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8xj5\" (UniqueName: \"kubernetes.io/projected/71ba531e-0810-4735-9466-d3410536a132-kube-api-access-r8xj5\") pod \"kuadrant-operator-controller-manager-55467d9f7-q4mm2\" (UID: \"71ba531e-0810-4735-9466-d3410536a132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:31.034347 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:31.034312 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ba531e-0810-4735-9466-d3410536a132-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-q4mm2\" (UID: \"71ba531e-0810-4735-9466-d3410536a132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:31.034535 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:31.034419 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8xj5\" (UniqueName: \"kubernetes.io/projected/71ba531e-0810-4735-9466-d3410536a132-kube-api-access-r8xj5\") pod \"kuadrant-operator-controller-manager-55467d9f7-q4mm2\" (UID: \"71ba531e-0810-4735-9466-d3410536a132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:31.034701 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:31.034681 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ba531e-0810-4735-9466-d3410536a132-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-q4mm2\" (UID: \"71ba531e-0810-4735-9466-d3410536a132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:31.044014 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:31.043985 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8xj5\" (UniqueName: \"kubernetes.io/projected/71ba531e-0810-4735-9466-d3410536a132-kube-api-access-r8xj5\") pod \"kuadrant-operator-controller-manager-55467d9f7-q4mm2\" (UID: \"71ba531e-0810-4735-9466-d3410536a132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:31.213215 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:31.213128 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:31.352405 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:31.352382 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2"] Jul 12 22:17:31.354323 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:17:31.354296 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ba531e_0810_4735_9466_d3410536a132.slice/crio-f3db7384d4ec23dca43c591181e1261cc0c49c1849f58aefd774b9b288021d91 WatchSource:0}: Error finding container f3db7384d4ec23dca43c591181e1261cc0c49c1849f58aefd774b9b288021d91: Status 404 returned error can't find the container with id f3db7384d4ec23dca43c591181e1261cc0c49c1849f58aefd774b9b288021d91 Jul 12 22:17:31.566131 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:31.566047 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" event={"ID":"71ba531e-0810-4735-9466-d3410536a132","Type":"ContainerStarted","Data":"f3db7384d4ec23dca43c591181e1261cc0c49c1849f58aefd774b9b288021d91"} Jul 12 22:17:33.429161 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:17:33.429074 2563 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-jck5g_kuadrant-system_ba3738f9-b1cb-4781-a9fd-84ad8cb693b0_0(3f6a51df407770a6d4f91fc0f97788b071abcd1e22d123bb9f91bab1cf9e3061): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-jck5g to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 12 22:17:33.429586 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:17:33.429254 2563 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-jck5g_kuadrant-system_ba3738f9-b1cb-4781-a9fd-84ad8cb693b0_0(3f6a51df407770a6d4f91fc0f97788b071abcd1e22d123bb9f91bab1cf9e3061): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-jck5g to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:33.583634 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:33.583593 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:33.592987 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:33.592951 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:33.595978 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:17:33.595941 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jck5g\" is forbidden: User \"system:node:ip-10-0-134-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-44.ec2.internal' and this object" podUID="ba3738f9-b1cb-4781-a9fd-84ad8cb693b0" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:33.656140 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:33.656050 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-extensions-socket-volume\") pod \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\" (UID: \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\") " Jul 12 22:17:33.656140 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:33.656125 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzds5\" (UniqueName: \"kubernetes.io/projected/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-kube-api-access-fzds5\") pod \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\" (UID: \"ba3738f9-b1cb-4781-a9fd-84ad8cb693b0\") " Jul 12 22:17:33.656429 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:33.656398 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ba3738f9-b1cb-4781-a9fd-84ad8cb693b0" (UID: "ba3738f9-b1cb-4781-a9fd-84ad8cb693b0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:17:33.658428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:33.658397 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-kube-api-access-fzds5" (OuterVolumeSpecName: "kube-api-access-fzds5") pod "ba3738f9-b1cb-4781-a9fd-84ad8cb693b0" (UID: "ba3738f9-b1cb-4781-a9fd-84ad8cb693b0"). InnerVolumeSpecName "kube-api-access-fzds5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:17:33.757332 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:33.757297 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-extensions-socket-volume\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:17:33.757332 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:33.757332 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzds5\" (UniqueName: \"kubernetes.io/projected/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0-kube-api-access-fzds5\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:17:34.589542 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:34.589506 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:34.592020 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:17:34.591976 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jck5g\" is forbidden: User \"system:node:ip-10-0-134-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-44.ec2.internal' and this object" podUID="ba3738f9-b1cb-4781-a9fd-84ad8cb693b0" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:34.598247 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:17:34.598216 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jck5g\" is forbidden: User \"system:node:ip-10-0-134-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-44.ec2.internal' and this object" podUID="ba3738f9-b1cb-4781-a9fd-84ad8cb693b0" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jck5g" Jul 12 22:17:34.632502 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:34.632468 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba3738f9-b1cb-4781-a9fd-84ad8cb693b0" path="/var/lib/kubelet/pods/ba3738f9-b1cb-4781-a9fd-84ad8cb693b0/volumes" Jul 12 22:17:35.596436 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:35.596403 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" event={"ID":"71ba531e-0810-4735-9466-d3410536a132","Type":"ContainerStarted","Data":"1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa"} Jul 12 22:17:35.596801 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:35.596528 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:35.616255 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:35.616205 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" podStartSLOduration=1.455164345 podStartE2EDuration="5.616188176s" totalImagesPullingTime="4.161023831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:17:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:17:35.614043635 +0000 UTC m=+437.482654460" watchObservedRunningTime="2026-07-12 22:17:35.616188176 +0000 UTC m=+437.484798999" Jul 12 22:17:46.603262 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:46.603181 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:59.353977 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.353942 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2"] Jul 12 22:17:59.354425 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.354144 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" podUID="71ba531e-0810-4735-9466-d3410536a132" containerName="manager" containerID="cri-o://1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa" gracePeriod=10 Jul 12 22:17:59.594060 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.594031 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:59.668329 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.668254 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ba531e-0810-4735-9466-d3410536a132-extensions-socket-volume\") pod \"71ba531e-0810-4735-9466-d3410536a132\" (UID: \"71ba531e-0810-4735-9466-d3410536a132\") " Jul 12 22:17:59.668507 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.668332 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8xj5\" (UniqueName: \"kubernetes.io/projected/71ba531e-0810-4735-9466-d3410536a132-kube-api-access-r8xj5\") pod \"71ba531e-0810-4735-9466-d3410536a132\" (UID: \"71ba531e-0810-4735-9466-d3410536a132\") " Jul 12 22:17:59.668776 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.668756 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71ba531e-0810-4735-9466-d3410536a132-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "71ba531e-0810-4735-9466-d3410536a132" (UID: "71ba531e-0810-4735-9466-d3410536a132"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:17:59.670451 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.670431 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71ba531e-0810-4735-9466-d3410536a132-kube-api-access-r8xj5" (OuterVolumeSpecName: "kube-api-access-r8xj5") pod "71ba531e-0810-4735-9466-d3410536a132" (UID: "71ba531e-0810-4735-9466-d3410536a132"). InnerVolumeSpecName "kube-api-access-r8xj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:17:59.704669 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.704633 2563 generic.go:356] "Generic (PLEG): container finished" podID="71ba531e-0810-4735-9466-d3410536a132" containerID="1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa" exitCode=0 Jul 12 22:17:59.704785 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.704684 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" Jul 12 22:17:59.704785 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.704683 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" event={"ID":"71ba531e-0810-4735-9466-d3410536a132","Type":"ContainerDied","Data":"1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa"} Jul 12 22:17:59.704785 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.704735 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2" event={"ID":"71ba531e-0810-4735-9466-d3410536a132","Type":"ContainerDied","Data":"f3db7384d4ec23dca43c591181e1261cc0c49c1849f58aefd774b9b288021d91"} Jul 12 22:17:59.704785 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.704764 2563 scope.go:117] "RemoveContainer" containerID="1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa" Jul 12 22:17:59.713804 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.713788 2563 scope.go:117] "RemoveContainer" containerID="1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa" Jul 12 22:17:59.714049 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:17:59.714030 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa\": container with ID starting with 1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa not found: ID does not exist" containerID="1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa" Jul 12 22:17:59.714099 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.714058 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa"} err="failed to get container status \"1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa\": rpc error: code = NotFound desc = could not find container \"1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa\": container with ID starting with 1690dcd084c93d459a6b5b4c0f0d09a4d468610089ee431e16aefda9ac9498fa not found: ID does not exist" Jul 12 22:17:59.726088 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.726061 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2"] Jul 12 22:17:59.729593 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.729567 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-q4mm2"] Jul 12 22:17:59.769005 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.768981 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71ba531e-0810-4735-9466-d3410536a132-extensions-socket-volume\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:17:59.769005 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:17:59.769003 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8xj5\" (UniqueName: \"kubernetes.io/projected/71ba531e-0810-4735-9466-d3410536a132-kube-api-access-r8xj5\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:18:00.630910 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:00.630871 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71ba531e-0810-4735-9466-d3410536a132" path="/var/lib/kubelet/pods/71ba531e-0810-4735-9466-d3410536a132/volumes" Jul 12 22:18:19.130035 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.130003 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vxkml"] Jul 12 22:18:19.130446 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.130280 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ba531e-0810-4735-9466-d3410536a132" containerName="manager" Jul 12 22:18:19.130446 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.130289 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ba531e-0810-4735-9466-d3410536a132" containerName="manager" Jul 12 22:18:19.130446 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.130344 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="71ba531e-0810-4735-9466-d3410536a132" containerName="manager" Jul 12 22:18:19.132852 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.132837 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:19.135293 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.135263 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 22:18:19.135420 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.135312 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 12 22:18:19.135420 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.135388 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 22:18:19.136216 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.136197 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qs8cq\"" Jul 12 22:18:19.145125 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.145103 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vxkml"] Jul 12 22:18:19.227329 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.227300 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vxkml"] Jul 12 22:18:19.320598 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.320562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcgr5\" (UniqueName: \"kubernetes.io/projected/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-kube-api-access-vcgr5\") pod \"limitador-limitador-64c94fd64f-vxkml\" (UID: \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:19.320783 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.320706 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-config-file\") pod \"limitador-limitador-64c94fd64f-vxkml\" (UID: \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:19.421679 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.421581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-config-file\") pod \"limitador-limitador-64c94fd64f-vxkml\" (UID: \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:19.421679 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.421629 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcgr5\" (UniqueName: \"kubernetes.io/projected/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-kube-api-access-vcgr5\") pod \"limitador-limitador-64c94fd64f-vxkml\" (UID: \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:19.422276 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.422254 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-config-file\") pod \"limitador-limitador-64c94fd64f-vxkml\" (UID: \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:19.430564 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.430536 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcgr5\" (UniqueName: \"kubernetes.io/projected/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-kube-api-access-vcgr5\") pod \"limitador-limitador-64c94fd64f-vxkml\" (UID: \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:19.442780 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.442752 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:19.582267 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.582238 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vxkml"] Jul 12 22:18:19.584736 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:18:19.584701 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998a5c8d_c5e1_47ac_83f6_f6f40019aa76.slice/crio-2ffe0be58708c7b9f7dc418b297a1a4e01f07b33c547534b61605cb9fd70ed8e WatchSource:0}: Error finding container 2ffe0be58708c7b9f7dc418b297a1a4e01f07b33c547534b61605cb9fd70ed8e: Status 404 returned error can't find the container with id 2ffe0be58708c7b9f7dc418b297a1a4e01f07b33c547534b61605cb9fd70ed8e Jul 12 22:18:19.800209 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.800128 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" event={"ID":"998a5c8d-c5e1-47ac-83f6-f6f40019aa76","Type":"ContainerStarted","Data":"2ffe0be58708c7b9f7dc418b297a1a4e01f07b33c547534b61605cb9fd70ed8e"} Jul 12 22:18:19.941506 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.941470 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6hbxn"] Jul 12 22:18:19.944020 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.943999 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6hbxn" Jul 12 22:18:19.946463 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.946444 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sswrt\"" Jul 12 22:18:19.949525 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:19.949503 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6hbxn"] Jul 12 22:18:20.127915 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:20.127874 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tg9h\" (UniqueName: \"kubernetes.io/projected/29a24763-1d83-41f8-9776-0f0599d1f3e9-kube-api-access-8tg9h\") pod \"authorino-58cd955675-6hbxn\" (UID: \"29a24763-1d83-41f8-9776-0f0599d1f3e9\") " pod="kuadrant-system/authorino-58cd955675-6hbxn" Jul 12 22:18:20.229283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:20.229244 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tg9h\" (UniqueName: \"kubernetes.io/projected/29a24763-1d83-41f8-9776-0f0599d1f3e9-kube-api-access-8tg9h\") pod \"authorino-58cd955675-6hbxn\" (UID: \"29a24763-1d83-41f8-9776-0f0599d1f3e9\") " pod="kuadrant-system/authorino-58cd955675-6hbxn" Jul 12 22:18:20.244149 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:20.244116 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tg9h\" (UniqueName: \"kubernetes.io/projected/29a24763-1d83-41f8-9776-0f0599d1f3e9-kube-api-access-8tg9h\") pod \"authorino-58cd955675-6hbxn\" (UID: \"29a24763-1d83-41f8-9776-0f0599d1f3e9\") " pod="kuadrant-system/authorino-58cd955675-6hbxn" Jul 12 22:18:20.254481 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:20.254447 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6hbxn" Jul 12 22:18:20.435230 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:20.435159 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6hbxn"] Jul 12 22:18:20.437465 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:18:20.437431 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29a24763_1d83_41f8_9776_0f0599d1f3e9.slice/crio-dcbd5647462efc6f1c4956fc4b6e973ead0cf60d5e14cbedaa1a12d69401ad7f WatchSource:0}: Error finding container dcbd5647462efc6f1c4956fc4b6e973ead0cf60d5e14cbedaa1a12d69401ad7f: Status 404 returned error can't find the container with id dcbd5647462efc6f1c4956fc4b6e973ead0cf60d5e14cbedaa1a12d69401ad7f Jul 12 22:18:20.806955 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:20.806871 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6hbxn" event={"ID":"29a24763-1d83-41f8-9776-0f0599d1f3e9","Type":"ContainerStarted","Data":"dcbd5647462efc6f1c4956fc4b6e973ead0cf60d5e14cbedaa1a12d69401ad7f"} Jul 12 22:18:24.518503 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:24.518414 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6hbxn"] Jul 12 22:18:24.836342 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:24.836245 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6hbxn" event={"ID":"29a24763-1d83-41f8-9776-0f0599d1f3e9","Type":"ContainerStarted","Data":"708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93"} Jul 12 22:18:24.837772 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:24.837748 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" event={"ID":"998a5c8d-c5e1-47ac-83f6-f6f40019aa76","Type":"ContainerStarted","Data":"c4e0773c5fe001a8494db3c45da731f2485ba506459133a446d1575c22c770fd"} Jul 12 22:18:24.837915 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:24.837879 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:24.849621 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:24.849578 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6hbxn" podStartSLOduration=2.165372881 podStartE2EDuration="5.849564465s" totalImagesPullingTime="3.684191584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:18:24.849051247 +0000 UTC m=+486.717662072" watchObservedRunningTime="2026-07-12 22:18:24.849564465 +0000 UTC m=+486.718175288" Jul 12 22:18:24.867340 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:24.867294 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" podStartSLOduration=1.329655725 podStartE2EDuration="5.867277663s" totalImagesPullingTime="4.537621938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:18:24.865125407 +0000 UTC m=+486.733736232" watchObservedRunningTime="2026-07-12 22:18:24.867277663 +0000 UTC m=+486.735888489" Jul 12 22:18:25.842218 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:25.842180 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6hbxn" podUID="29a24763-1d83-41f8-9776-0f0599d1f3e9" containerName="authorino" containerID="cri-o://708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93" gracePeriod=30 Jul 12 22:18:26.087497 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.087475 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6hbxn" Jul 12 22:18:26.184762 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.184671 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tg9h\" (UniqueName: \"kubernetes.io/projected/29a24763-1d83-41f8-9776-0f0599d1f3e9-kube-api-access-8tg9h\") pod \"29a24763-1d83-41f8-9776-0f0599d1f3e9\" (UID: \"29a24763-1d83-41f8-9776-0f0599d1f3e9\") " Jul 12 22:18:26.186843 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.186810 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a24763-1d83-41f8-9776-0f0599d1f3e9-kube-api-access-8tg9h" (OuterVolumeSpecName: "kube-api-access-8tg9h") pod "29a24763-1d83-41f8-9776-0f0599d1f3e9" (UID: "29a24763-1d83-41f8-9776-0f0599d1f3e9"). InnerVolumeSpecName "kube-api-access-8tg9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:18:26.285835 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.285799 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tg9h\" (UniqueName: \"kubernetes.io/projected/29a24763-1d83-41f8-9776-0f0599d1f3e9-kube-api-access-8tg9h\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:18:26.847677 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.847633 2563 generic.go:356] "Generic (PLEG): container finished" podID="29a24763-1d83-41f8-9776-0f0599d1f3e9" containerID="708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93" exitCode=0 Jul 12 22:18:26.847677 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.847673 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6hbxn" Jul 12 22:18:26.848127 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.847689 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6hbxn" event={"ID":"29a24763-1d83-41f8-9776-0f0599d1f3e9","Type":"ContainerDied","Data":"708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93"} Jul 12 22:18:26.848127 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.847728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6hbxn" event={"ID":"29a24763-1d83-41f8-9776-0f0599d1f3e9","Type":"ContainerDied","Data":"dcbd5647462efc6f1c4956fc4b6e973ead0cf60d5e14cbedaa1a12d69401ad7f"} Jul 12 22:18:26.848127 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.847747 2563 scope.go:117] "RemoveContainer" containerID="708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93" Jul 12 22:18:26.856172 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.856153 2563 scope.go:117] "RemoveContainer" containerID="708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93" Jul 12 22:18:26.856439 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:18:26.856419 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93\": container with ID starting with 708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93 not found: ID does not exist" containerID="708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93" Jul 12 22:18:26.856518 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.856447 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93"} err="failed to get container status \"708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93\": rpc error: code = NotFound desc = could not find container \"708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93\": container with ID starting with 708a0088cdba139f2d2582cd7b94bb11994ed47aa1e231cb819c317458c9ec93 not found: ID does not exist" Jul 12 22:18:26.862904 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.862880 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6hbxn"] Jul 12 22:18:26.866354 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:26.866331 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6hbxn"] Jul 12 22:18:28.631103 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:28.631068 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29a24763-1d83-41f8-9776-0f0599d1f3e9" path="/var/lib/kubelet/pods/29a24763-1d83-41f8-9776-0f0599d1f3e9/volumes" Jul 12 22:18:34.040267 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:34.040229 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vxkml"] Jul 12 22:18:34.040801 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:34.040417 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" podUID="998a5c8d-c5e1-47ac-83f6-f6f40019aa76" containerName="limitador" containerID="cri-o://c4e0773c5fe001a8494db3c45da731f2485ba506459133a446d1575c22c770fd" gracePeriod=30 Jul 12 22:18:34.041158 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:34.041136 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:34.887270 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:34.887239 2563 generic.go:356] "Generic (PLEG): container finished" podID="998a5c8d-c5e1-47ac-83f6-f6f40019aa76" containerID="c4e0773c5fe001a8494db3c45da731f2485ba506459133a446d1575c22c770fd" exitCode=0 Jul 12 22:18:34.887468 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:34.887306 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" event={"ID":"998a5c8d-c5e1-47ac-83f6-f6f40019aa76","Type":"ContainerDied","Data":"c4e0773c5fe001a8494db3c45da731f2485ba506459133a446d1575c22c770fd"} Jul 12 22:18:34.979426 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:34.979402 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:35.050745 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.050714 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-config-file\") pod \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\" (UID: \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\") " Jul 12 22:18:35.051110 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.050831 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcgr5\" (UniqueName: \"kubernetes.io/projected/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-kube-api-access-vcgr5\") pod \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\" (UID: \"998a5c8d-c5e1-47ac-83f6-f6f40019aa76\") " Jul 12 22:18:35.051110 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.051088 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-config-file" (OuterVolumeSpecName: "config-file") pod "998a5c8d-c5e1-47ac-83f6-f6f40019aa76" (UID: "998a5c8d-c5e1-47ac-83f6-f6f40019aa76"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:18:35.053021 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.052998 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-kube-api-access-vcgr5" (OuterVolumeSpecName: "kube-api-access-vcgr5") pod "998a5c8d-c5e1-47ac-83f6-f6f40019aa76" (UID: "998a5c8d-c5e1-47ac-83f6-f6f40019aa76"). InnerVolumeSpecName "kube-api-access-vcgr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:18:35.151958 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.151876 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcgr5\" (UniqueName: \"kubernetes.io/projected/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-kube-api-access-vcgr5\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:18:35.151958 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.151904 2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/998a5c8d-c5e1-47ac-83f6-f6f40019aa76-config-file\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:18:35.893433 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.893398 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" event={"ID":"998a5c8d-c5e1-47ac-83f6-f6f40019aa76","Type":"ContainerDied","Data":"2ffe0be58708c7b9f7dc418b297a1a4e01f07b33c547534b61605cb9fd70ed8e"} Jul 12 22:18:35.893433 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.893437 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vxkml" Jul 12 22:18:35.893658 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.893488 2563 scope.go:117] "RemoveContainer" containerID="c4e0773c5fe001a8494db3c45da731f2485ba506459133a446d1575c22c770fd" Jul 12 22:18:35.916873 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.916847 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vxkml"] Jul 12 22:18:35.920791 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:35.920769 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vxkml"] Jul 12 22:18:36.631329 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:36.631297 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="998a5c8d-c5e1-47ac-83f6-f6f40019aa76" path="/var/lib/kubelet/pods/998a5c8d-c5e1-47ac-83f6-f6f40019aa76/volumes" Jul 12 22:18:37.015432 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.015342 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd"] Jul 12 22:18:37.015683 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.015669 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="998a5c8d-c5e1-47ac-83f6-f6f40019aa76" containerName="limitador" Jul 12 22:18:37.015726 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.015685 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="998a5c8d-c5e1-47ac-83f6-f6f40019aa76" containerName="limitador" Jul 12 22:18:37.015726 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.015696 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a24763-1d83-41f8-9776-0f0599d1f3e9" containerName="authorino" Jul 12 22:18:37.015726 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.015705 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a24763-1d83-41f8-9776-0f0599d1f3e9" containerName="authorino" Jul 12 22:18:37.015811 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.015762 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="998a5c8d-c5e1-47ac-83f6-f6f40019aa76" containerName="limitador" Jul 12 22:18:37.015811 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.015774 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a24763-1d83-41f8-9776-0f0599d1f3e9" containerName="authorino" Jul 12 22:18:37.020079 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.020063 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:37.023736 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.023717 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 12 22:18:37.023834 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.023747 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 12 22:18:37.024641 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.024626 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fv4df\"" Jul 12 22:18:37.024688 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.024632 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 22:18:37.034745 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.034723 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd"] Jul 12 22:18:37.065621 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.065592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxmmc\" (UniqueName: \"kubernetes.io/projected/9a510e49-449c-4e63-a9a5-94fae278cd96-kube-api-access-gxmmc\") pod \"postgres-69fb4594cf-8v6pd\" (UID: \"9a510e49-449c-4e63-a9a5-94fae278cd96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:37.065782 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.065656 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9a510e49-449c-4e63-a9a5-94fae278cd96-data\") pod \"postgres-69fb4594cf-8v6pd\" (UID: \"9a510e49-449c-4e63-a9a5-94fae278cd96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:37.166759 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.166722 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9a510e49-449c-4e63-a9a5-94fae278cd96-data\") pod \"postgres-69fb4594cf-8v6pd\" (UID: \"9a510e49-449c-4e63-a9a5-94fae278cd96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:37.166927 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.166779 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxmmc\" (UniqueName: \"kubernetes.io/projected/9a510e49-449c-4e63-a9a5-94fae278cd96-kube-api-access-gxmmc\") pod \"postgres-69fb4594cf-8v6pd\" (UID: \"9a510e49-449c-4e63-a9a5-94fae278cd96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:37.167105 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.167086 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9a510e49-449c-4e63-a9a5-94fae278cd96-data\") pod \"postgres-69fb4594cf-8v6pd\" (UID: \"9a510e49-449c-4e63-a9a5-94fae278cd96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:37.191611 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.191570 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxmmc\" (UniqueName: \"kubernetes.io/projected/9a510e49-449c-4e63-a9a5-94fae278cd96-kube-api-access-gxmmc\") pod \"postgres-69fb4594cf-8v6pd\" (UID: \"9a510e49-449c-4e63-a9a5-94fae278cd96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:37.330020 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.329938 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:37.462304 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.462281 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd"] Jul 12 22:18:37.464647 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:18:37.464624 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a510e49_449c_4e63_a9a5_94fae278cd96.slice/crio-33ffcf9faf14bb1235b8a70cfd236e1732d08677b217985c7aa8d7a3d3ca4cbc WatchSource:0}: Error finding container 33ffcf9faf14bb1235b8a70cfd236e1732d08677b217985c7aa8d7a3d3ca4cbc: Status 404 returned error can't find the container with id 33ffcf9faf14bb1235b8a70cfd236e1732d08677b217985c7aa8d7a3d3ca4cbc Jul 12 22:18:37.905726 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:37.905689 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" event={"ID":"9a510e49-449c-4e63-a9a5-94fae278cd96","Type":"ContainerStarted","Data":"33ffcf9faf14bb1235b8a70cfd236e1732d08677b217985c7aa8d7a3d3ca4cbc"} Jul 12 22:18:42.933260 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:42.933219 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" event={"ID":"9a510e49-449c-4e63-a9a5-94fae278cd96","Type":"ContainerStarted","Data":"ea3ea19c49dd0da6c4ccb2eb8b4ba7603cf85d49e75a62fbbd6f58d33c93d969"} Jul 12 22:18:42.933666 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:42.933331 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:42.948844 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:42.948783 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" podStartSLOduration=2.285914712 podStartE2EDuration="6.948769035s" totalImagesPullingTime="4.662854323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:18:42.94802689 +0000 UTC m=+504.816637717" watchObservedRunningTime="2026-07-12 22:18:42.948769035 +0000 UTC m=+504.817379889" Jul 12 22:18:48.967130 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:48.967095 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8v6pd" Jul 12 22:18:56.262599 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.262564 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59d5b88fb4-p9md6"] Jul 12 22:18:56.283787 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.283756 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59d5b88fb4-p9md6"] Jul 12 22:18:56.283944 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.283868 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:18:56.286356 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.286326 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 12 22:18:56.286533 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.286387 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 12 22:18:56.286533 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.286420 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pg98b\"" Jul 12 22:18:56.414616 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.414581 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d8865b84c-hcvmf"] Jul 12 22:18:56.417099 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.417077 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:18:56.427467 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.427442 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8865b84c-hcvmf"] Jul 12 22:18:56.430163 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.430139 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xktfn\" (UniqueName: \"kubernetes.io/projected/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-kube-api-access-xktfn\") pod \"maas-controller-59d5b88fb4-p9md6\" (UID: \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\") " pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:18:56.430287 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.430192 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-webhook-cert\") pod \"maas-controller-59d5b88fb4-p9md6\" (UID: \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\") " pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:18:56.531515 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.531407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbf59\" (UniqueName: \"kubernetes.io/projected/a2305de2-a7d2-4791-95d0-0d0041ffbc23-kube-api-access-fbf59\") pod \"maas-controller-d8865b84c-hcvmf\" (UID: \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\") " pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:18:56.531515 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.531471 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-webhook-cert\") pod \"maas-controller-59d5b88fb4-p9md6\" (UID: \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\") " pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:18:56.531747 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.531555 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xktfn\" (UniqueName: \"kubernetes.io/projected/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-kube-api-access-xktfn\") pod \"maas-controller-59d5b88fb4-p9md6\" (UID: \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\") " pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:18:56.531747 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.531585 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a2305de2-a7d2-4791-95d0-0d0041ffbc23-webhook-cert\") pod \"maas-controller-d8865b84c-hcvmf\" (UID: \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\") " pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:18:56.531747 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:18:56.531620 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 12 22:18:56.531747 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:18:56.531693 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-webhook-cert podName:1c74dbbd-508b-4454-8fa0-8332e6cc7bfd nodeName:}" failed. No retries permitted until 2026-07-12 22:18:57.031675484 +0000 UTC m=+518.900286286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-webhook-cert") pod "maas-controller-59d5b88fb4-p9md6" (UID: "1c74dbbd-508b-4454-8fa0-8332e6cc7bfd") : secret "maas-controller-webhook-cert" not found Jul 12 22:18:56.540670 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.540638 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xktfn\" (UniqueName: \"kubernetes.io/projected/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-kube-api-access-xktfn\") pod \"maas-controller-59d5b88fb4-p9md6\" (UID: \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\") " pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:18:56.631991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.631957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbf59\" (UniqueName: \"kubernetes.io/projected/a2305de2-a7d2-4791-95d0-0d0041ffbc23-kube-api-access-fbf59\") pod \"maas-controller-d8865b84c-hcvmf\" (UID: \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\") " pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:18:56.632167 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.632058 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a2305de2-a7d2-4791-95d0-0d0041ffbc23-webhook-cert\") pod \"maas-controller-d8865b84c-hcvmf\" (UID: \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\") " pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:18:56.632228 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:18:56.632162 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 12 22:18:56.632228 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:18:56.632225 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2305de2-a7d2-4791-95d0-0d0041ffbc23-webhook-cert podName:a2305de2-a7d2-4791-95d0-0d0041ffbc23 nodeName:}" failed. No retries permitted until 2026-07-12 22:18:57.132206931 +0000 UTC m=+519.000817733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/a2305de2-a7d2-4791-95d0-0d0041ffbc23-webhook-cert") pod "maas-controller-d8865b84c-hcvmf" (UID: "a2305de2-a7d2-4791-95d0-0d0041ffbc23") : secret "maas-controller-webhook-cert" not found Jul 12 22:18:56.642401 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:56.642358 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbf59\" (UniqueName: \"kubernetes.io/projected/a2305de2-a7d2-4791-95d0-0d0041ffbc23-kube-api-access-fbf59\") pod \"maas-controller-d8865b84c-hcvmf\" (UID: \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\") " pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:18:57.034645 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:57.034600 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-webhook-cert\") pod \"maas-controller-59d5b88fb4-p9md6\" (UID: \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\") " pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:18:57.037217 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:57.037195 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-webhook-cert\") pod \"maas-controller-59d5b88fb4-p9md6\" (UID: \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\") " pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:18:57.135622 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:57.135585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a2305de2-a7d2-4791-95d0-0d0041ffbc23-webhook-cert\") pod \"maas-controller-d8865b84c-hcvmf\" (UID: \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\") " pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:18:57.138085 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:57.138055 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a2305de2-a7d2-4791-95d0-0d0041ffbc23-webhook-cert\") pod \"maas-controller-d8865b84c-hcvmf\" (UID: \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\") " pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:18:57.194820 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:57.194788 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:18:57.327743 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:57.327654 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:18:57.329189 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:57.329161 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59d5b88fb4-p9md6"] Jul 12 22:18:57.330640 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:18:57.330618 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c74dbbd_508b_4454_8fa0_8332e6cc7bfd.slice/crio-b5c8fd073678258af4c3c16adc61deca9f7decb70b8e910aad26b8cf4788c89d WatchSource:0}: Error finding container b5c8fd073678258af4c3c16adc61deca9f7decb70b8e910aad26b8cf4788c89d: Status 404 returned error can't find the container with id b5c8fd073678258af4c3c16adc61deca9f7decb70b8e910aad26b8cf4788c89d Jul 12 22:18:57.460616 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:57.460589 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8865b84c-hcvmf"] Jul 12 22:18:57.462051 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:18:57.462019 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2305de2_a7d2_4791_95d0_0d0041ffbc23.slice/crio-6981895f60bfe1fbcd03aee36fdecb4e842e640cc824ffe85f8e21fbd0c68234 WatchSource:0}: Error finding container 6981895f60bfe1fbcd03aee36fdecb4e842e640cc824ffe85f8e21fbd0c68234: Status 404 returned error can't find the container with id 6981895f60bfe1fbcd03aee36fdecb4e842e640cc824ffe85f8e21fbd0c68234 Jul 12 22:18:58.019668 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:58.019470 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" event={"ID":"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd","Type":"ContainerStarted","Data":"b5c8fd073678258af4c3c16adc61deca9f7decb70b8e910aad26b8cf4788c89d"} Jul 12 22:18:58.021282 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:18:58.021212 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8865b84c-hcvmf" event={"ID":"a2305de2-a7d2-4791-95d0-0d0041ffbc23","Type":"ContainerStarted","Data":"6981895f60bfe1fbcd03aee36fdecb4e842e640cc824ffe85f8e21fbd0c68234"} Jul 12 22:19:00.033558 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:00.033518 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" event={"ID":"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd","Type":"ContainerStarted","Data":"87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf"} Jul 12 22:19:00.034031 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:00.033625 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:19:00.037436 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:00.035630 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8865b84c-hcvmf" event={"ID":"a2305de2-a7d2-4791-95d0-0d0041ffbc23","Type":"ContainerStarted","Data":"0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec"} Jul 12 22:19:00.037436 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:00.036110 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:19:00.053420 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:00.053359 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" podStartSLOduration=2.035663002 podStartE2EDuration="4.053344578s" totalImagesPullingTime="2.017681576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:19:00.050901836 +0000 UTC m=+521.919512660" watchObservedRunningTime="2026-07-12 22:19:00.053344578 +0000 UTC m=+521.921955406" Jul 12 22:19:11.043687 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:11.043608 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:19:11.062424 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:11.062351 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d8865b84c-hcvmf" podStartSLOduration=13.179692455 podStartE2EDuration="15.062334124s" totalImagesPullingTime="1.882641669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:19:00.077040437 +0000 UTC m=+521.945651260" watchObservedRunningTime="2026-07-12 22:19:11.062334124 +0000 UTC m=+532.930945002" Jul 12 22:19:12.048431 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.048401 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:19:12.101517 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.101487 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59d5b88fb4-p9md6"] Jul 12 22:19:12.101683 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.101658 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" podUID="1c74dbbd-508b-4454-8fa0-8332e6cc7bfd" containerName="manager" containerID="cri-o://87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf" gracePeriod=30 Jul 12 22:19:12.337254 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.337229 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:19:12.466207 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.466176 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xktfn\" (UniqueName: \"kubernetes.io/projected/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-kube-api-access-xktfn\") pod \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\" (UID: \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\") " Jul 12 22:19:12.466396 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.466252 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-webhook-cert\") pod \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\" (UID: \"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd\") " Jul 12 22:19:12.468649 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.468611 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-kube-api-access-xktfn" (OuterVolumeSpecName: "kube-api-access-xktfn") pod "1c74dbbd-508b-4454-8fa0-8332e6cc7bfd" (UID: "1c74dbbd-508b-4454-8fa0-8332e6cc7bfd"). InnerVolumeSpecName "kube-api-access-xktfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:19:12.468649 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.468623 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1c74dbbd-508b-4454-8fa0-8332e6cc7bfd" (UID: "1c74dbbd-508b-4454-8fa0-8332e6cc7bfd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:19:12.567361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.567321 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xktfn\" (UniqueName: \"kubernetes.io/projected/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-kube-api-access-xktfn\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:19:12.567361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:12.567353 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd-webhook-cert\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:19:13.100990 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:13.100955 2563 generic.go:356] "Generic (PLEG): container finished" podID="1c74dbbd-508b-4454-8fa0-8332e6cc7bfd" containerID="87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf" exitCode=0 Jul 12 22:19:13.101477 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:13.101002 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" Jul 12 22:19:13.101477 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:13.101029 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" event={"ID":"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd","Type":"ContainerDied","Data":"87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf"} Jul 12 22:19:13.101477 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:13.101084 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59d5b88fb4-p9md6" event={"ID":"1c74dbbd-508b-4454-8fa0-8332e6cc7bfd","Type":"ContainerDied","Data":"b5c8fd073678258af4c3c16adc61deca9f7decb70b8e910aad26b8cf4788c89d"} Jul 12 22:19:13.101477 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:13.101133 2563 scope.go:117] "RemoveContainer" containerID="87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf" Jul 12 22:19:13.110189 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:13.110171 2563 scope.go:117] "RemoveContainer" containerID="87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf" Jul 12 22:19:13.110505 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:19:13.110484 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf\": container with ID starting with 87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf not found: ID does not exist" containerID="87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf" Jul 12 22:19:13.110549 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:13.110514 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf"} err="failed to get container status \"87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf\": rpc error: code = NotFound desc = could not find container \"87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf\": container with ID starting with 87c83450c37abab2f8efa463711b0dcf97c34f24d69b17e02b87a06b40b70fbf not found: ID does not exist" Jul 12 22:19:13.118340 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:13.118315 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59d5b88fb4-p9md6"] Jul 12 22:19:13.121835 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:13.121812 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59d5b88fb4-p9md6"] Jul 12 22:19:14.630978 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:14.630943 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c74dbbd-508b-4454-8fa0-8332e6cc7bfd" path="/var/lib/kubelet/pods/1c74dbbd-508b-4454-8fa0-8332e6cc7bfd/volumes" Jul 12 22:19:31.734500 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.734467 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn"] Jul 12 22:19:31.735072 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.734775 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c74dbbd-508b-4454-8fa0-8332e6cc7bfd" containerName="manager" Jul 12 22:19:31.735072 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.734785 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c74dbbd-508b-4454-8fa0-8332e6cc7bfd" containerName="manager" Jul 12 22:19:31.735072 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.734851 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c74dbbd-508b-4454-8fa0-8332e6cc7bfd" containerName="manager" Jul 12 22:19:31.739357 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.739337 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:31.742012 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.741995 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 12 22:19:31.742088 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.742062 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-t26l9\"" Jul 12 22:19:31.750056 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.750029 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn"] Jul 12 22:19:31.824914 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.824878 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6860ac2c-c2bb-45b3-b21c-464ca22f35cf-openshift-service-ca\") pod \"maas-api-74ffd49cf8-ln7tn\" (UID: \"6860ac2c-c2bb-45b3-b21c-464ca22f35cf\") " pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:31.825099 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.824933 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6860ac2c-c2bb-45b3-b21c-464ca22f35cf-maas-api-tls\") pod \"maas-api-74ffd49cf8-ln7tn\" (UID: \"6860ac2c-c2bb-45b3-b21c-464ca22f35cf\") " pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:31.825099 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.824973 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxk76\" (UniqueName: \"kubernetes.io/projected/6860ac2c-c2bb-45b3-b21c-464ca22f35cf-kube-api-access-bxk76\") pod \"maas-api-74ffd49cf8-ln7tn\" (UID: \"6860ac2c-c2bb-45b3-b21c-464ca22f35cf\") " pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:31.926104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.926052 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6860ac2c-c2bb-45b3-b21c-464ca22f35cf-maas-api-tls\") pod \"maas-api-74ffd49cf8-ln7tn\" (UID: \"6860ac2c-c2bb-45b3-b21c-464ca22f35cf\") " pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:31.926104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.926109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxk76\" (UniqueName: \"kubernetes.io/projected/6860ac2c-c2bb-45b3-b21c-464ca22f35cf-kube-api-access-bxk76\") pod \"maas-api-74ffd49cf8-ln7tn\" (UID: \"6860ac2c-c2bb-45b3-b21c-464ca22f35cf\") " pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:31.926421 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.926156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6860ac2c-c2bb-45b3-b21c-464ca22f35cf-openshift-service-ca\") pod \"maas-api-74ffd49cf8-ln7tn\" (UID: \"6860ac2c-c2bb-45b3-b21c-464ca22f35cf\") " pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:31.926811 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.926790 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6860ac2c-c2bb-45b3-b21c-464ca22f35cf-openshift-service-ca\") pod \"maas-api-74ffd49cf8-ln7tn\" (UID: \"6860ac2c-c2bb-45b3-b21c-464ca22f35cf\") " pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:31.928643 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.928625 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6860ac2c-c2bb-45b3-b21c-464ca22f35cf-maas-api-tls\") pod \"maas-api-74ffd49cf8-ln7tn\" (UID: \"6860ac2c-c2bb-45b3-b21c-464ca22f35cf\") " pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:31.935741 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:31.935719 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxk76\" (UniqueName: \"kubernetes.io/projected/6860ac2c-c2bb-45b3-b21c-464ca22f35cf-kube-api-access-bxk76\") pod \"maas-api-74ffd49cf8-ln7tn\" (UID: \"6860ac2c-c2bb-45b3-b21c-464ca22f35cf\") " pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:32.051570 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.051474 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:32.196398 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.196353 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn"] Jul 12 22:19:32.197248 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:19:32.197222 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6860ac2c_c2bb_45b3_b21c_464ca22f35cf.slice/crio-c38cb78d524b83bcb7f980b645955861184f1d76f1cfce5f332baaded4c53152 WatchSource:0}: Error finding container c38cb78d524b83bcb7f980b645955861184f1d76f1cfce5f332baaded4c53152: Status 404 returned error can't find the container with id c38cb78d524b83bcb7f980b645955861184f1d76f1cfce5f332baaded4c53152 Jul 12 22:19:32.305439 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.305356 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-dsjgn"] Jul 12 22:19:32.309609 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.309585 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:32.312049 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.312025 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-p8frm\"" Jul 12 22:19:32.312049 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.312040 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 12 22:19:32.327922 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.327892 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-dsjgn"] Jul 12 22:19:32.329223 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.329204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c82806ac-a7c2-4f98-9663-d8cacfd2e32a-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-dsjgn\" (UID: \"c82806ac-a7c2-4f98-9663-d8cacfd2e32a\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:32.329298 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.329266 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvzgm\" (UniqueName: \"kubernetes.io/projected/c82806ac-a7c2-4f98-9663-d8cacfd2e32a-kube-api-access-bvzgm\") pod \"payload-processing-f9b8d9b9b-dsjgn\" (UID: \"c82806ac-a7c2-4f98-9663-d8cacfd2e32a\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:32.430325 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.430288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvzgm\" (UniqueName: \"kubernetes.io/projected/c82806ac-a7c2-4f98-9663-d8cacfd2e32a-kube-api-access-bvzgm\") pod \"payload-processing-f9b8d9b9b-dsjgn\" (UID: \"c82806ac-a7c2-4f98-9663-d8cacfd2e32a\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:32.430510 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.430347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c82806ac-a7c2-4f98-9663-d8cacfd2e32a-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-dsjgn\" (UID: \"c82806ac-a7c2-4f98-9663-d8cacfd2e32a\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:32.431257 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.431231 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c82806ac-a7c2-4f98-9663-d8cacfd2e32a-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-dsjgn\" (UID: \"c82806ac-a7c2-4f98-9663-d8cacfd2e32a\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:32.439526 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.439494 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvzgm\" (UniqueName: \"kubernetes.io/projected/c82806ac-a7c2-4f98-9663-d8cacfd2e32a-kube-api-access-bvzgm\") pod \"payload-processing-f9b8d9b9b-dsjgn\" (UID: \"c82806ac-a7c2-4f98-9663-d8cacfd2e32a\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:32.626719 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.626683 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:32.767416 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:32.767362 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-dsjgn"] Jul 12 22:19:32.769450 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:19:32.769423 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82806ac_a7c2_4f98_9663_d8cacfd2e32a.slice/crio-b5c3c7771029f1732e3c054e2e8c1c958548fdd6e0289b692c5f6cf32f1d7ad3 WatchSource:0}: Error finding container b5c3c7771029f1732e3c054e2e8c1c958548fdd6e0289b692c5f6cf32f1d7ad3: Status 404 returned error can't find the container with id b5c3c7771029f1732e3c054e2e8c1c958548fdd6e0289b692c5f6cf32f1d7ad3 Jul 12 22:19:33.204635 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:33.204601 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" event={"ID":"c82806ac-a7c2-4f98-9663-d8cacfd2e32a","Type":"ContainerStarted","Data":"b5c3c7771029f1732e3c054e2e8c1c958548fdd6e0289b692c5f6cf32f1d7ad3"} Jul 12 22:19:33.205899 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:33.205869 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" event={"ID":"6860ac2c-c2bb-45b3-b21c-464ca22f35cf","Type":"ContainerStarted","Data":"c38cb78d524b83bcb7f980b645955861184f1d76f1cfce5f332baaded4c53152"} Jul 12 22:19:34.212108 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:34.212072 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" event={"ID":"6860ac2c-c2bb-45b3-b21c-464ca22f35cf","Type":"ContainerStarted","Data":"2a86ab86652cdc08e3fe9f11cb7a6218af89ec4237e44609c4438521c57e3f2e"} Jul 12 22:19:34.212549 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:34.212253 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:34.230002 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:34.229947 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" podStartSLOduration=1.859847631 podStartE2EDuration="3.229932376s" totalImagesPullingTime="1.370084745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:19:34.228325779 +0000 UTC m=+556.096936618" watchObservedRunningTime="2026-07-12 22:19:34.229932376 +0000 UTC m=+556.098543268" Jul 12 22:19:35.218715 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:35.218680 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" event={"ID":"c82806ac-a7c2-4f98-9663-d8cacfd2e32a","Type":"ContainerStarted","Data":"0d7e07c5a782011aa156fa7517bd32ddd77a9412c9e8271e10a6731c19f68db3"} Jul 12 22:19:35.219134 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:35.218773 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:35.237256 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:35.237207 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" podStartSLOduration=1.745536563 podStartE2EDuration="3.237191621s" totalImagesPullingTime="1.491655058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:19:35.234524678 +0000 UTC m=+557.103135502" watchObservedRunningTime="2026-07-12 22:19:35.237191621 +0000 UTC m=+557.105802445" Jul 12 22:19:40.224248 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:40.224220 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-74ffd49cf8-ln7tn" Jul 12 22:19:46.226048 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:46.226017 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-dsjgn" Jul 12 22:19:51.280254 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.278535 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx"] Jul 12 22:19:51.283703 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.283678 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.286423 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.286397 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 22:19:51.287236 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.287215 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 12 22:19:51.287343 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.287258 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 22:19:51.287343 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.287304 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7867w\"" Jul 12 22:19:51.289318 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.289292 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx"] Jul 12 22:19:51.355130 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.355103 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.355279 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.355163 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.355279 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.355192 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3b6cdf-5dff-4305-a75b-aa447323ccff-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.355279 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.355238 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q6fv\" (UniqueName: \"kubernetes.io/projected/ed3b6cdf-5dff-4305-a75b-aa447323ccff-kube-api-access-7q6fv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.355413 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.355286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.355413 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.355315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.455728 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.455693 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.455728 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.455732 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.455961 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.455749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3b6cdf-5dff-4305-a75b-aa447323ccff-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.455961 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.455783 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q6fv\" (UniqueName: \"kubernetes.io/projected/ed3b6cdf-5dff-4305-a75b-aa447323ccff-kube-api-access-7q6fv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.455961 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.455806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.455961 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.455830 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.456169 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.456142 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.456222 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.456149 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.456270 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.456219 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.458224 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.458201 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed3b6cdf-5dff-4305-a75b-aa447323ccff-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.458502 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.458486 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed3b6cdf-5dff-4305-a75b-aa447323ccff-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.463674 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.463654 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q6fv\" (UniqueName: \"kubernetes.io/projected/ed3b6cdf-5dff-4305-a75b-aa447323ccff-kube-api-access-7q6fv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx\" (UID: \"ed3b6cdf-5dff-4305-a75b-aa447323ccff\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.595635 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.595607 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:19:51.726558 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:51.726531 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx"] Jul 12 22:19:51.728185 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:19:51.728160 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3b6cdf_5dff_4305_a75b_aa447323ccff.slice/crio-25a2484d56248205a2ff04bc3c5c68e9ed42094edc03b70abd28f48dc7e83ef6 WatchSource:0}: Error finding container 25a2484d56248205a2ff04bc3c5c68e9ed42094edc03b70abd28f48dc7e83ef6: Status 404 returned error can't find the container with id 25a2484d56248205a2ff04bc3c5c68e9ed42094edc03b70abd28f48dc7e83ef6 Jul 12 22:19:52.305914 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:52.305879 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" event={"ID":"ed3b6cdf-5dff-4305-a75b-aa447323ccff","Type":"ContainerStarted","Data":"25a2484d56248205a2ff04bc3c5c68e9ed42094edc03b70abd28f48dc7e83ef6"} Jul 12 22:19:53.312648 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:53.312605 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" event={"ID":"ed3b6cdf-5dff-4305-a75b-aa447323ccff","Type":"ContainerStarted","Data":"d8014861fce946c7f00597df22b5dae874bf5ded9290386094488840366a70c6"} Jul 12 22:19:53.331768 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:19:53.331718 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" podStartSLOduration=1.326743784 podStartE2EDuration="2.331700731s" totalImagesPullingTime="1.004956947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:19:53.329901621 +0000 UTC m=+575.198512446" watchObservedRunningTime="2026-07-12 22:19:53.331700731 +0000 UTC m=+575.200311556" Jul 12 22:20:01.596652 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:01.596602 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:20:01.597060 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:01.596676 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:20:01.609343 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:01.609316 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:20:02.367583 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:02.367556 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx" Jul 12 22:20:18.597119 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:18.597092 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:20:18.598018 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:18.597995 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:20:18.603656 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:18.603638 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:20:18.604816 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:18.604797 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:20:31.683607 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.683571 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g"] Jul 12 22:20:31.689543 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.689524 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.691859 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.691837 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 12 22:20:31.697513 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.697492 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g"] Jul 12 22:20:31.795504 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.795473 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.795504 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.795507 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.795708 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.795558 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.795708 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.795659 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.795708 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.795688 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.795814 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.795706 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts4qt\" (UniqueName: \"kubernetes.io/projected/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-kube-api-access-ts4qt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.896287 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.896248 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.896287 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.896286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.896501 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.896306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts4qt\" (UniqueName: \"kubernetes.io/projected/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-kube-api-access-ts4qt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.896501 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.896360 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.896501 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.896408 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.896501 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.896465 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.896768 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.896736 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.896954 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.896824 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.896954 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.896869 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.898746 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.898723 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.899126 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.899103 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:31.904586 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:31.904565 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts4qt\" (UniqueName: \"kubernetes.io/projected/64d626a9-fcfb-42f4-8422-33bc8dc75cb2-kube-api-access-ts4qt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gd22g\" (UID: \"64d626a9-fcfb-42f4-8422-33bc8dc75cb2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:32.000896 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:32.000812 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:32.145000 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:32.140006 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g"] Jul 12 22:20:32.145000 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:32.142587 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:20:32.504921 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:32.504885 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" event={"ID":"64d626a9-fcfb-42f4-8422-33bc8dc75cb2","Type":"ContainerStarted","Data":"05fd124eb2b3e4d9a67349e54b7fd4f16317b0e5ee1f4ea891e49aa7fc6a84fd"} Jul 12 22:20:32.505071 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:32.504945 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" event={"ID":"64d626a9-fcfb-42f4-8422-33bc8dc75cb2","Type":"ContainerStarted","Data":"d261bb3e2469defbb7413d64b42ac08ba0e802d4cbc3f09d89e3af122a6e5fdc"} Jul 12 22:20:32.520857 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:32.520801 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" podStartSLOduration=1.324180017 podStartE2EDuration="1.520784158s" totalImagesPullingTime="196.604141ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:20:32.52050107 +0000 UTC m=+614.389111894" watchObservedRunningTime="2026-07-12 22:20:32.520784158 +0000 UTC m=+614.389394983" Jul 12 22:20:42.001919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:42.001831 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:42.001919 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:42.001909 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:42.014262 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:42.014235 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:20:42.564252 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:20:42.564228 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gd22g" Jul 12 22:21:02.278009 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.277976 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt"] Jul 12 22:21:02.281175 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.281149 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.284769 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.284732 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 12 22:21:02.290455 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.290430 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt"] Jul 12 22:21:02.336755 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.336727 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.336889 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.336787 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.336889 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.336864 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/972b4f9b-23be-491c-aeaf-92190e88c90e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.336963 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.336899 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdgrh\" (UniqueName: \"kubernetes.io/projected/972b4f9b-23be-491c-aeaf-92190e88c90e-kube-api-access-gdgrh\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.336997 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.336957 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.336997 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.336980 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.437900 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.437870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/972b4f9b-23be-491c-aeaf-92190e88c90e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.438036 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.437904 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdgrh\" (UniqueName: \"kubernetes.io/projected/972b4f9b-23be-491c-aeaf-92190e88c90e-kube-api-access-gdgrh\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.438036 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.437941 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.438036 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.437966 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.438036 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.438010 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.438220 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.438057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.438433 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.438411 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.438537 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.438475 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.438537 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.438516 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.440406 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.440385 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/972b4f9b-23be-491c-aeaf-92190e88c90e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.440486 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.440469 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/972b4f9b-23be-491c-aeaf-92190e88c90e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.445767 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.445744 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdgrh\" (UniqueName: \"kubernetes.io/projected/972b4f9b-23be-491c-aeaf-92190e88c90e-kube-api-access-gdgrh\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt\" (UID: \"972b4f9b-23be-491c-aeaf-92190e88c90e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.590824 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.590792 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:02.728497 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:02.728234 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt"] Jul 12 22:21:03.660558 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:03.660525 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" event={"ID":"972b4f9b-23be-491c-aeaf-92190e88c90e","Type":"ContainerStarted","Data":"a694e76713afeb721f97255bba56d3a272aee533a296415303633bb14ae6699d"} Jul 12 22:21:03.660558 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:03.660563 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" event={"ID":"972b4f9b-23be-491c-aeaf-92190e88c90e","Type":"ContainerStarted","Data":"645a221ef990f70fa7713d2fcca43167512d7f7eb01e93325f3de4011a900121"} Jul 12 22:21:03.678750 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:03.678705 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" podStartSLOduration=1.514460851 podStartE2EDuration="1.678692139s" totalImagesPullingTime="164.231288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:21:03.676896784 +0000 UTC m=+645.545507600" watchObservedRunningTime="2026-07-12 22:21:03.678692139 +0000 UTC m=+645.547302963" Jul 12 22:21:12.591214 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:12.591171 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:12.591649 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:12.591247 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:12.603684 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:12.603661 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:21:12.716030 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:21:12.715999 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt" Jul 12 22:23:30.294265 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:30.294224 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d8865b84c-hcvmf"] Jul 12 22:23:30.294812 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:30.294440 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d8865b84c-hcvmf" podUID="a2305de2-a7d2-4791-95d0-0d0041ffbc23" containerName="manager" containerID="cri-o://0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec" gracePeriod=30 Jul 12 22:23:30.544212 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:30.544184 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:23:30.654761 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:30.654728 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a2305de2-a7d2-4791-95d0-0d0041ffbc23-webhook-cert\") pod \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\" (UID: \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\") " Jul 12 22:23:30.654921 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:30.654798 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbf59\" (UniqueName: \"kubernetes.io/projected/a2305de2-a7d2-4791-95d0-0d0041ffbc23-kube-api-access-fbf59\") pod \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\" (UID: \"a2305de2-a7d2-4791-95d0-0d0041ffbc23\") " Jul 12 22:23:30.656810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:30.656779 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2305de2-a7d2-4791-95d0-0d0041ffbc23-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a2305de2-a7d2-4791-95d0-0d0041ffbc23" (UID: "a2305de2-a7d2-4791-95d0-0d0041ffbc23"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:23:30.656950 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:30.656837 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2305de2-a7d2-4791-95d0-0d0041ffbc23-kube-api-access-fbf59" (OuterVolumeSpecName: "kube-api-access-fbf59") pod "a2305de2-a7d2-4791-95d0-0d0041ffbc23" (UID: "a2305de2-a7d2-4791-95d0-0d0041ffbc23"). InnerVolumeSpecName "kube-api-access-fbf59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:23:30.756133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:30.756104 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbf59\" (UniqueName: \"kubernetes.io/projected/a2305de2-a7d2-4791-95d0-0d0041ffbc23-kube-api-access-fbf59\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:23:30.756133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:30.756130 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a2305de2-a7d2-4791-95d0-0d0041ffbc23-webhook-cert\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:23:31.391833 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:31.391800 2563 generic.go:356] "Generic (PLEG): container finished" podID="a2305de2-a7d2-4791-95d0-0d0041ffbc23" containerID="0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec" exitCode=0 Jul 12 22:23:31.392230 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:31.391856 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8865b84c-hcvmf" Jul 12 22:23:31.392230 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:31.391888 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8865b84c-hcvmf" event={"ID":"a2305de2-a7d2-4791-95d0-0d0041ffbc23","Type":"ContainerDied","Data":"0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec"} Jul 12 22:23:31.392230 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:31.391941 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8865b84c-hcvmf" event={"ID":"a2305de2-a7d2-4791-95d0-0d0041ffbc23","Type":"ContainerDied","Data":"6981895f60bfe1fbcd03aee36fdecb4e842e640cc824ffe85f8e21fbd0c68234"} Jul 12 22:23:31.392230 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:31.391966 2563 scope.go:117] "RemoveContainer" containerID="0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec" Jul 12 22:23:31.401059 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:31.401041 2563 scope.go:117] "RemoveContainer" containerID="0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec" Jul 12 22:23:31.401320 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:23:31.401303 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec\": container with ID starting with 0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec not found: ID does not exist" containerID="0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec" Jul 12 22:23:31.401363 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:31.401328 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec"} err="failed to get container status \"0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec\": rpc error: code = NotFound desc = could not find container \"0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec\": container with ID starting with 0a87718b0902acf73212f1f0029122c7cdd9389d00f6bc369a1f485340c2e0ec not found: ID does not exist" Jul 12 22:23:31.414552 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:31.414528 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d8865b84c-hcvmf"] Jul 12 22:23:31.419349 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:31.419331 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d8865b84c-hcvmf"] Jul 12 22:23:32.176267 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.176234 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d8865b84c-gjw92"] Jul 12 22:23:32.176584 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.176571 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2305de2-a7d2-4791-95d0-0d0041ffbc23" containerName="manager" Jul 12 22:23:32.176625 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.176586 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2305de2-a7d2-4791-95d0-0d0041ffbc23" containerName="manager" Jul 12 22:23:32.176660 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.176646 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2305de2-a7d2-4791-95d0-0d0041ffbc23" containerName="manager" Jul 12 22:23:32.180712 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.180696 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:23:32.183990 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.183970 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 12 22:23:32.184104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.183998 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pg98b\"" Jul 12 22:23:32.184104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.183978 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 12 22:23:32.189492 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.189468 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8865b84c-gjw92"] Jul 12 22:23:32.268716 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.268687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb8903d6-da56-485b-bbbe-98ebc583b80f-webhook-cert\") pod \"maas-controller-d8865b84c-gjw92\" (UID: \"fb8903d6-da56-485b-bbbe-98ebc583b80f\") " pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:23:32.268854 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.268752 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snj8t\" (UniqueName: \"kubernetes.io/projected/fb8903d6-da56-485b-bbbe-98ebc583b80f-kube-api-access-snj8t\") pod \"maas-controller-d8865b84c-gjw92\" (UID: \"fb8903d6-da56-485b-bbbe-98ebc583b80f\") " pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:23:32.369100 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.369069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb8903d6-da56-485b-bbbe-98ebc583b80f-webhook-cert\") pod \"maas-controller-d8865b84c-gjw92\" (UID: \"fb8903d6-da56-485b-bbbe-98ebc583b80f\") " pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:23:32.369233 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.369132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snj8t\" (UniqueName: \"kubernetes.io/projected/fb8903d6-da56-485b-bbbe-98ebc583b80f-kube-api-access-snj8t\") pod \"maas-controller-d8865b84c-gjw92\" (UID: \"fb8903d6-da56-485b-bbbe-98ebc583b80f\") " pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:23:32.371678 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.371659 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb8903d6-da56-485b-bbbe-98ebc583b80f-webhook-cert\") pod \"maas-controller-d8865b84c-gjw92\" (UID: \"fb8903d6-da56-485b-bbbe-98ebc583b80f\") " pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:23:32.377543 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.377524 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snj8t\" (UniqueName: \"kubernetes.io/projected/fb8903d6-da56-485b-bbbe-98ebc583b80f-kube-api-access-snj8t\") pod \"maas-controller-d8865b84c-gjw92\" (UID: \"fb8903d6-da56-485b-bbbe-98ebc583b80f\") " pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:23:32.492351 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.492265 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:23:32.629268 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:23:32.629241 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8903d6_da56_485b_bbbe_98ebc583b80f.slice/crio-fd23dbd74821b901760368b43e6e6dd1b7edbb73caee1e69148109feb4cfbd05 WatchSource:0}: Error finding container fd23dbd74821b901760368b43e6e6dd1b7edbb73caee1e69148109feb4cfbd05: Status 404 returned error can't find the container with id fd23dbd74821b901760368b43e6e6dd1b7edbb73caee1e69148109feb4cfbd05 Jul 12 22:23:32.632179 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.632150 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2305de2-a7d2-4791-95d0-0d0041ffbc23" path="/var/lib/kubelet/pods/a2305de2-a7d2-4791-95d0-0d0041ffbc23/volumes" Jul 12 22:23:32.632541 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:32.632525 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8865b84c-gjw92"] Jul 12 22:23:33.404142 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:33.404110 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8865b84c-gjw92" event={"ID":"fb8903d6-da56-485b-bbbe-98ebc583b80f","Type":"ContainerStarted","Data":"77384fc44675c996fdc79928aea3ac81ae375917f787b5c9490117d623856c9e"} Jul 12 22:23:33.404309 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:33.404161 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8865b84c-gjw92" event={"ID":"fb8903d6-da56-485b-bbbe-98ebc583b80f","Type":"ContainerStarted","Data":"fd23dbd74821b901760368b43e6e6dd1b7edbb73caee1e69148109feb4cfbd05"} Jul 12 22:23:33.404309 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:33.404183 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:23:33.426807 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:33.426760 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d8865b84c-gjw92" podStartSLOduration=1.050052281 podStartE2EDuration="1.426745142s" totalImagesPullingTime="376.692861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:23:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:23:33.425083236 +0000 UTC m=+795.293694061" watchObservedRunningTime="2026-07-12 22:23:33.426745142 +0000 UTC m=+795.295355966" Jul 12 22:23:44.410865 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:23:44.410793 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d8865b84c-gjw92" Jul 12 22:25:18.631991 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:25:18.631922 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:25:18.632536 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:25:18.632174 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:25:18.638992 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:25:18.638971 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:25:18.639187 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:25:18.639174 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:30:18.663614 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:30:18.663582 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:30:18.665979 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:30:18.665955 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:30:18.669875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:30:18.669857 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:30:18.672409 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:30:18.672394 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:35:18.695479 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:18.695445 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:35:18.699680 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:18.699655 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:35:18.702075 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:18.702054 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:35:18.705856 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:18.705837 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:35:56.706578 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.706547 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp"] Jul 12 22:35:56.710241 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.710217 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:56.712476 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.712448 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-cbe7f53b-serving-cert\"" Jul 12 22:35:56.722747 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.722724 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp"] Jul 12 22:35:56.773257 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.773220 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf2635-cd6e-4efc-bc2a-f3094eef959a-maas-api-tls\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:56.773485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.773281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bgsx\" (UniqueName: \"kubernetes.io/projected/6abf2635-cd6e-4efc-bc2a-f3094eef959a-kube-api-access-7bgsx\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:56.773485 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.773402 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf2635-cd6e-4efc-bc2a-f3094eef959a-openshift-service-ca\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:56.874103 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.874065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf2635-cd6e-4efc-bc2a-f3094eef959a-openshift-service-ca\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:56.874292 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.874115 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf2635-cd6e-4efc-bc2a-f3094eef959a-maas-api-tls\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:56.874292 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.874156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bgsx\" (UniqueName: \"kubernetes.io/projected/6abf2635-cd6e-4efc-bc2a-f3094eef959a-kube-api-access-7bgsx\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:56.874292 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:35:56.874278 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-serving-cert: secret "maas-api-e2e-ait-cbe7f53b-serving-cert" not found Jul 12 22:35:56.874476 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:35:56.874402 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6abf2635-cd6e-4efc-bc2a-f3094eef959a-maas-api-tls podName:6abf2635-cd6e-4efc-bc2a-f3094eef959a nodeName:}" failed. No retries permitted until 2026-07-12 22:35:57.374346226 +0000 UTC m=+1539.242957031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6abf2635-cd6e-4efc-bc2a-f3094eef959a-maas-api-tls") pod "maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" (UID: "6abf2635-cd6e-4efc-bc2a-f3094eef959a") : secret "maas-api-e2e-ait-cbe7f53b-serving-cert" not found Jul 12 22:35:56.874825 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.874800 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf2635-cd6e-4efc-bc2a-f3094eef959a-openshift-service-ca\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:56.887996 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:56.887972 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bgsx\" (UniqueName: \"kubernetes.io/projected/6abf2635-cd6e-4efc-bc2a-f3094eef959a-kube-api-access-7bgsx\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:57.379453 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:57.379410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf2635-cd6e-4efc-bc2a-f3094eef959a-maas-api-tls\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:57.382144 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:57.382117 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf2635-cd6e-4efc-bc2a-f3094eef959a-maas-api-tls\") pod \"maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:57.621544 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:57.621504 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:57.762363 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:57.762334 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp"] Jul 12 22:35:57.763896 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:35:57.763870 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6abf2635_cd6e_4efc_bc2a_f3094eef959a.slice/crio-9ca663336561cb9ac586fd98aa019bfb290a71ae7e2e0031c4cdcc83bc56fbfc WatchSource:0}: Error finding container 9ca663336561cb9ac586fd98aa019bfb290a71ae7e2e0031c4cdcc83bc56fbfc: Status 404 returned error can't find the container with id 9ca663336561cb9ac586fd98aa019bfb290a71ae7e2e0031c4cdcc83bc56fbfc Jul 12 22:35:57.765141 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:57.765125 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:35:58.093800 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:58.093767 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" event={"ID":"6abf2635-cd6e-4efc-bc2a-f3094eef959a","Type":"ContainerStarted","Data":"9ca663336561cb9ac586fd98aa019bfb290a71ae7e2e0031c4cdcc83bc56fbfc"} Jul 12 22:35:59.100603 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:59.100508 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" event={"ID":"6abf2635-cd6e-4efc-bc2a-f3094eef959a","Type":"ContainerStarted","Data":"9bf5bd20f85e24f762d7b1cd6fd144a5bf760a7096e03186da5cac3638bc7cea"} Jul 12 22:35:59.100603 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:59.100584 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:35:59.119255 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:35:59.119195 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" podStartSLOduration=2.753246056 podStartE2EDuration="3.119178794s" totalImagesPullingTime="365.932738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:35:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:35:59.117015256 +0000 UTC m=+1540.985626080" watchObservedRunningTime="2026-07-12 22:35:59.119178794 +0000 UTC m=+1540.987789618" Jul 12 22:36:05.110476 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:36:05.110445 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:37:02.205621 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.205577 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp"] Jul 12 22:37:02.206166 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.205832 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" podUID="6abf2635-cd6e-4efc-bc2a-f3094eef959a" containerName="maas-api" containerID="cri-o://9bf5bd20f85e24f762d7b1cd6fd144a5bf760a7096e03186da5cac3638bc7cea" gracePeriod=30 Jul 12 22:37:02.429949 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.429916 2563 generic.go:356] "Generic (PLEG): container finished" podID="6abf2635-cd6e-4efc-bc2a-f3094eef959a" containerID="9bf5bd20f85e24f762d7b1cd6fd144a5bf760a7096e03186da5cac3638bc7cea" exitCode=0 Jul 12 22:37:02.430114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.429970 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" event={"ID":"6abf2635-cd6e-4efc-bc2a-f3094eef959a","Type":"ContainerDied","Data":"9bf5bd20f85e24f762d7b1cd6fd144a5bf760a7096e03186da5cac3638bc7cea"} Jul 12 22:37:02.457169 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.457114 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:37:02.573083 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.573052 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bgsx\" (UniqueName: \"kubernetes.io/projected/6abf2635-cd6e-4efc-bc2a-f3094eef959a-kube-api-access-7bgsx\") pod \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " Jul 12 22:37:02.573265 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.573129 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf2635-cd6e-4efc-bc2a-f3094eef959a-openshift-service-ca\") pod \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " Jul 12 22:37:02.573265 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.573149 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf2635-cd6e-4efc-bc2a-f3094eef959a-maas-api-tls\") pod \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\" (UID: \"6abf2635-cd6e-4efc-bc2a-f3094eef959a\") " Jul 12 22:37:02.573503 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.573481 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6abf2635-cd6e-4efc-bc2a-f3094eef959a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6abf2635-cd6e-4efc-bc2a-f3094eef959a" (UID: "6abf2635-cd6e-4efc-bc2a-f3094eef959a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:37:02.575478 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.575437 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6abf2635-cd6e-4efc-bc2a-f3094eef959a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6abf2635-cd6e-4efc-bc2a-f3094eef959a" (UID: "6abf2635-cd6e-4efc-bc2a-f3094eef959a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:37:02.575561 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.575478 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6abf2635-cd6e-4efc-bc2a-f3094eef959a-kube-api-access-7bgsx" (OuterVolumeSpecName: "kube-api-access-7bgsx") pod "6abf2635-cd6e-4efc-bc2a-f3094eef959a" (UID: "6abf2635-cd6e-4efc-bc2a-f3094eef959a"). InnerVolumeSpecName "kube-api-access-7bgsx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:37:02.674077 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.674033 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bgsx\" (UniqueName: \"kubernetes.io/projected/6abf2635-cd6e-4efc-bc2a-f3094eef959a-kube-api-access-7bgsx\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:37:02.674077 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.674068 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf2635-cd6e-4efc-bc2a-f3094eef959a-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:37:02.674077 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:02.674080 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf2635-cd6e-4efc-bc2a-f3094eef959a-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:37:03.436800 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:03.436768 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" Jul 12 22:37:03.436800 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:03.436792 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp" event={"ID":"6abf2635-cd6e-4efc-bc2a-f3094eef959a","Type":"ContainerDied","Data":"9ca663336561cb9ac586fd98aa019bfb290a71ae7e2e0031c4cdcc83bc56fbfc"} Jul 12 22:37:03.437323 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:03.436858 2563 scope.go:117] "RemoveContainer" containerID="9bf5bd20f85e24f762d7b1cd6fd144a5bf760a7096e03186da5cac3638bc7cea" Jul 12 22:37:03.453283 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:03.453251 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp"] Jul 12 22:37:03.456721 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:03.456694 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cbe7f53b-6cfd8448b8-2clvp"] Jul 12 22:37:04.631684 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:04.631651 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6abf2635-cd6e-4efc-bc2a-f3094eef959a" path="/var/lib/kubelet/pods/6abf2635-cd6e-4efc-bc2a-f3094eef959a/volumes" Jul 12 22:37:19.000713 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.000673 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx"] Jul 12 22:37:19.001092 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.001084 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6abf2635-cd6e-4efc-bc2a-f3094eef959a" containerName="maas-api" Jul 12 22:37:19.001141 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.001096 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6abf2635-cd6e-4efc-bc2a-f3094eef959a" containerName="maas-api" Jul 12 22:37:19.001182 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.001152 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6abf2635-cd6e-4efc-bc2a-f3094eef959a" containerName="maas-api" Jul 12 22:37:19.004282 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.004265 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.006864 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.006815 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e6750e7c-serving-cert\"" Jul 12 22:37:19.013189 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.013161 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx"] Jul 12 22:37:19.121663 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.121622 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62780bf4-9441-4a29-81fa-7d92cab6dfec-openshift-service-ca\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.121849 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.121756 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62780bf4-9441-4a29-81fa-7d92cab6dfec-maas-api-tls\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.121849 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.121792 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqb9v\" (UniqueName: \"kubernetes.io/projected/62780bf4-9441-4a29-81fa-7d92cab6dfec-kube-api-access-mqb9v\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.222888 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.222852 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62780bf4-9441-4a29-81fa-7d92cab6dfec-maas-api-tls\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.222888 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.222890 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqb9v\" (UniqueName: \"kubernetes.io/projected/62780bf4-9441-4a29-81fa-7d92cab6dfec-kube-api-access-mqb9v\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.223121 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.222942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62780bf4-9441-4a29-81fa-7d92cab6dfec-openshift-service-ca\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.223121 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:37:19.223018 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-serving-cert: secret "maas-api-e2e-migrate-e6750e7c-serving-cert" not found Jul 12 22:37:19.223121 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:37:19.223087 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62780bf4-9441-4a29-81fa-7d92cab6dfec-maas-api-tls podName:62780bf4-9441-4a29-81fa-7d92cab6dfec nodeName:}" failed. No retries permitted until 2026-07-12 22:37:19.723069803 +0000 UTC m=+1621.591680604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/62780bf4-9441-4a29-81fa-7d92cab6dfec-maas-api-tls") pod "maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" (UID: "62780bf4-9441-4a29-81fa-7d92cab6dfec") : secret "maas-api-e2e-migrate-e6750e7c-serving-cert" not found Jul 12 22:37:19.223584 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.223567 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62780bf4-9441-4a29-81fa-7d92cab6dfec-openshift-service-ca\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.231724 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.231701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqb9v\" (UniqueName: \"kubernetes.io/projected/62780bf4-9441-4a29-81fa-7d92cab6dfec-kube-api-access-mqb9v\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.726659 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.726613 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62780bf4-9441-4a29-81fa-7d92cab6dfec-maas-api-tls\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.729299 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.729277 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62780bf4-9441-4a29-81fa-7d92cab6dfec-maas-api-tls\") pod \"maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:19.915207 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:19.915177 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:20.056861 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:20.056837 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx"] Jul 12 22:37:20.058501 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:37:20.058476 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62780bf4_9441_4a29_81fa_7d92cab6dfec.slice/crio-652686871e51e5b50ec174367aa73fc28cd09ef4fae4caaf9d59f38dd9be9583 WatchSource:0}: Error finding container 652686871e51e5b50ec174367aa73fc28cd09ef4fae4caaf9d59f38dd9be9583: Status 404 returned error can't find the container with id 652686871e51e5b50ec174367aa73fc28cd09ef4fae4caaf9d59f38dd9be9583 Jul 12 22:37:20.525814 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:20.525782 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" event={"ID":"62780bf4-9441-4a29-81fa-7d92cab6dfec","Type":"ContainerStarted","Data":"652686871e51e5b50ec174367aa73fc28cd09ef4fae4caaf9d59f38dd9be9583"} Jul 12 22:37:21.531950 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:21.531915 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" event={"ID":"62780bf4-9441-4a29-81fa-7d92cab6dfec","Type":"ContainerStarted","Data":"3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599"} Jul 12 22:37:21.532358 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:21.532042 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:37:21.548821 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:21.548767 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" podStartSLOduration=3.118901967 podStartE2EDuration="3.548751118s" totalImagesPullingTime="429.849151ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:37:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:37:21.546495359 +0000 UTC m=+1623.415106183" watchObservedRunningTime="2026-07-12 22:37:21.548751118 +0000 UTC m=+1623.417361941" Jul 12 22:37:27.543895 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:37:27.543865 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:38:28.878517 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:28.878480 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx"] Jul 12 22:38:28.879091 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:28.878733 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" podUID="62780bf4-9441-4a29-81fa-7d92cab6dfec" containerName="maas-api" containerID="cri-o://3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599" gracePeriod=30 Jul 12 22:38:29.138111 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.138088 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:38:29.241558 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.241512 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62780bf4-9441-4a29-81fa-7d92cab6dfec-maas-api-tls\") pod \"62780bf4-9441-4a29-81fa-7d92cab6dfec\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " Jul 12 22:38:29.241740 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.241596 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62780bf4-9441-4a29-81fa-7d92cab6dfec-openshift-service-ca\") pod \"62780bf4-9441-4a29-81fa-7d92cab6dfec\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " Jul 12 22:38:29.241740 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.241624 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqb9v\" (UniqueName: \"kubernetes.io/projected/62780bf4-9441-4a29-81fa-7d92cab6dfec-kube-api-access-mqb9v\") pod \"62780bf4-9441-4a29-81fa-7d92cab6dfec\" (UID: \"62780bf4-9441-4a29-81fa-7d92cab6dfec\") " Jul 12 22:38:29.241987 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.241959 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62780bf4-9441-4a29-81fa-7d92cab6dfec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62780bf4-9441-4a29-81fa-7d92cab6dfec" (UID: "62780bf4-9441-4a29-81fa-7d92cab6dfec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:38:29.243857 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.243818 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62780bf4-9441-4a29-81fa-7d92cab6dfec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62780bf4-9441-4a29-81fa-7d92cab6dfec" (UID: "62780bf4-9441-4a29-81fa-7d92cab6dfec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:38:29.243952 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.243877 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62780bf4-9441-4a29-81fa-7d92cab6dfec-kube-api-access-mqb9v" (OuterVolumeSpecName: "kube-api-access-mqb9v") pod "62780bf4-9441-4a29-81fa-7d92cab6dfec" (UID: "62780bf4-9441-4a29-81fa-7d92cab6dfec"). InnerVolumeSpecName "kube-api-access-mqb9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:38:29.342912 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.342870 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62780bf4-9441-4a29-81fa-7d92cab6dfec-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:38:29.342912 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.342904 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62780bf4-9441-4a29-81fa-7d92cab6dfec-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:38:29.342912 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.342916 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqb9v\" (UniqueName: \"kubernetes.io/projected/62780bf4-9441-4a29-81fa-7d92cab6dfec-kube-api-access-mqb9v\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:38:29.880467 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.880427 2563 generic.go:356] "Generic (PLEG): container finished" podID="62780bf4-9441-4a29-81fa-7d92cab6dfec" containerID="3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599" exitCode=0 Jul 12 22:38:29.880967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.880475 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" event={"ID":"62780bf4-9441-4a29-81fa-7d92cab6dfec","Type":"ContainerDied","Data":"3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599"} Jul 12 22:38:29.880967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.880496 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" Jul 12 22:38:29.880967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.880551 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx" event={"ID":"62780bf4-9441-4a29-81fa-7d92cab6dfec","Type":"ContainerDied","Data":"652686871e51e5b50ec174367aa73fc28cd09ef4fae4caaf9d59f38dd9be9583"} Jul 12 22:38:29.880967 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.880588 2563 scope.go:117] "RemoveContainer" containerID="3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599" Jul 12 22:38:29.891145 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.891123 2563 scope.go:117] "RemoveContainer" containerID="3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599" Jul 12 22:38:29.891467 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:38:29.891446 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599\": container with ID starting with 3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599 not found: ID does not exist" containerID="3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599" Jul 12 22:38:29.891570 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.891484 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599"} err="failed to get container status \"3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599\": rpc error: code = NotFound desc = could not find container \"3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599\": container with ID starting with 3a99e187bc452e7f1ad1d155553dcaa4f6c03226d9ef80cd1f72b8c2d35aa599 not found: ID does not exist" Jul 12 22:38:29.903244 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.903211 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx"] Jul 12 22:38:29.908988 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:29.908960 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6750e7c-7b964457cd-z7tjx"] Jul 12 22:38:30.630509 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:30.630474 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62780bf4-9441-4a29-81fa-7d92cab6dfec" path="/var/lib/kubelet/pods/62780bf4-9441-4a29-81fa-7d92cab6dfec/volumes" Jul 12 22:38:36.400388 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.400344 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld"] Jul 12 22:38:36.400848 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.400713 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62780bf4-9441-4a29-81fa-7d92cab6dfec" containerName="maas-api" Jul 12 22:38:36.400848 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.400726 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62780bf4-9441-4a29-81fa-7d92cab6dfec" containerName="maas-api" Jul 12 22:38:36.400848 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.400783 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="62780bf4-9441-4a29-81fa-7d92cab6dfec" containerName="maas-api" Jul 12 22:38:36.405114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.405085 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.407576 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.407545 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c725b4f0-serving-cert\"" Jul 12 22:38:36.417118 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.417087 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld"] Jul 12 22:38:36.504536 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.504484 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9jr7\" (UniqueName: \"kubernetes.io/projected/94ef04eb-b50e-4e8f-afef-b080e6ad5314-kube-api-access-j9jr7\") pod \"maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.504713 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.504559 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef04eb-b50e-4e8f-afef-b080e6ad5314-maas-api-tls\") pod \"maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.504713 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.504618 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef04eb-b50e-4e8f-afef-b080e6ad5314-openshift-service-ca\") pod \"maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.605659 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.605624 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9jr7\" (UniqueName: \"kubernetes.io/projected/94ef04eb-b50e-4e8f-afef-b080e6ad5314-kube-api-access-j9jr7\") pod \"maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.605845 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.605671 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef04eb-b50e-4e8f-afef-b080e6ad5314-maas-api-tls\") pod \"maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.605845 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.605716 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef04eb-b50e-4e8f-afef-b080e6ad5314-openshift-service-ca\") pod \"maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.606474 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.606454 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef04eb-b50e-4e8f-afef-b080e6ad5314-openshift-service-ca\") pod \"maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.608574 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.608555 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef04eb-b50e-4e8f-afef-b080e6ad5314-maas-api-tls\") pod \"maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.618951 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.618920 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9jr7\" (UniqueName: \"kubernetes.io/projected/94ef04eb-b50e-4e8f-afef-b080e6ad5314-kube-api-access-j9jr7\") pod \"maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.719340 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.719243 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:36.863526 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.863501 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld"] Jul 12 22:38:36.865868 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:38:36.865838 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ef04eb_b50e_4e8f_afef_b080e6ad5314.slice/crio-b5eb6a2b4f6bc3c7de88a1c95f2834575dd39f7a113c1f2b0f292650dc7f1f32 WatchSource:0}: Error finding container b5eb6a2b4f6bc3c7de88a1c95f2834575dd39f7a113c1f2b0f292650dc7f1f32: Status 404 returned error can't find the container with id b5eb6a2b4f6bc3c7de88a1c95f2834575dd39f7a113c1f2b0f292650dc7f1f32 Jul 12 22:38:36.919944 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:36.919910 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" event={"ID":"94ef04eb-b50e-4e8f-afef-b080e6ad5314","Type":"ContainerStarted","Data":"b5eb6a2b4f6bc3c7de88a1c95f2834575dd39f7a113c1f2b0f292650dc7f1f32"} Jul 12 22:38:37.927000 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:37.926906 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" event={"ID":"94ef04eb-b50e-4e8f-afef-b080e6ad5314","Type":"ContainerStarted","Data":"dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8"} Jul 12 22:38:37.927000 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:37.926972 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:37.947525 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:37.947454 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" podStartSLOduration=1.157252076 podStartE2EDuration="1.947437578s" totalImagesPullingTime="790.185502ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:38:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:38:37.947266666 +0000 UTC m=+1699.815877491" watchObservedRunningTime="2026-07-12 22:38:37.947437578 +0000 UTC m=+1699.816048402" Jul 12 22:38:43.938163 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:43.938070 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:52.738425 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:52.738388 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld"] Jul 12 22:38:52.738968 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:52.738611 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" podUID="94ef04eb-b50e-4e8f-afef-b080e6ad5314" containerName="maas-api" containerID="cri-o://dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8" gracePeriod=30 Jul 12 22:38:53.009692 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.009667 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:53.012012 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.011988 2563 generic.go:356] "Generic (PLEG): container finished" podID="94ef04eb-b50e-4e8f-afef-b080e6ad5314" containerID="dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8" exitCode=0 Jul 12 22:38:53.012128 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.012021 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" Jul 12 22:38:53.012128 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.012034 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" event={"ID":"94ef04eb-b50e-4e8f-afef-b080e6ad5314","Type":"ContainerDied","Data":"dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8"} Jul 12 22:38:53.012128 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.012074 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld" event={"ID":"94ef04eb-b50e-4e8f-afef-b080e6ad5314","Type":"ContainerDied","Data":"b5eb6a2b4f6bc3c7de88a1c95f2834575dd39f7a113c1f2b0f292650dc7f1f32"} Jul 12 22:38:53.012128 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.012095 2563 scope.go:117] "RemoveContainer" containerID="dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8" Jul 12 22:38:53.020507 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.020487 2563 scope.go:117] "RemoveContainer" containerID="dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8" Jul 12 22:38:53.020777 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:38:53.020751 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8\": container with ID starting with dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8 not found: ID does not exist" containerID="dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8" Jul 12 22:38:53.020882 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.020789 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8"} err="failed to get container status \"dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8\": rpc error: code = NotFound desc = could not find container \"dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8\": container with ID starting with dc6511aaa09910e713778c6c2dd57ccecac065f2da4a62a1cdf366f44e1c18e8 not found: ID does not exist" Jul 12 22:38:53.152005 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.151976 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef04eb-b50e-4e8f-afef-b080e6ad5314-openshift-service-ca\") pod \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " Jul 12 22:38:53.152191 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.152089 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9jr7\" (UniqueName: \"kubernetes.io/projected/94ef04eb-b50e-4e8f-afef-b080e6ad5314-kube-api-access-j9jr7\") pod \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " Jul 12 22:38:53.152191 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.152136 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef04eb-b50e-4e8f-afef-b080e6ad5314-maas-api-tls\") pod \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\" (UID: \"94ef04eb-b50e-4e8f-afef-b080e6ad5314\") " Jul 12 22:38:53.152321 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.152297 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94ef04eb-b50e-4e8f-afef-b080e6ad5314-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "94ef04eb-b50e-4e8f-afef-b080e6ad5314" (UID: "94ef04eb-b50e-4e8f-afef-b080e6ad5314"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:38:53.154234 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.154210 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ef04eb-b50e-4e8f-afef-b080e6ad5314-kube-api-access-j9jr7" (OuterVolumeSpecName: "kube-api-access-j9jr7") pod "94ef04eb-b50e-4e8f-afef-b080e6ad5314" (UID: "94ef04eb-b50e-4e8f-afef-b080e6ad5314"). InnerVolumeSpecName "kube-api-access-j9jr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:38:53.154234 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.154218 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ef04eb-b50e-4e8f-afef-b080e6ad5314-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "94ef04eb-b50e-4e8f-afef-b080e6ad5314" (UID: "94ef04eb-b50e-4e8f-afef-b080e6ad5314"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:38:53.253439 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.253331 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9jr7\" (UniqueName: \"kubernetes.io/projected/94ef04eb-b50e-4e8f-afef-b080e6ad5314-kube-api-access-j9jr7\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:38:53.253439 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.253363 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef04eb-b50e-4e8f-afef-b080e6ad5314-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:38:53.253439 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.253400 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef04eb-b50e-4e8f-afef-b080e6ad5314-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:38:53.332893 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.332862 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld"] Jul 12 22:38:53.336704 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:53.336683 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c725b4f0-659b654f4-kl2ld"] Jul 12 22:38:54.631347 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:38:54.631315 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94ef04eb-b50e-4e8f-afef-b080e6ad5314" path="/var/lib/kubelet/pods/94ef04eb-b50e-4e8f-afef-b080e6ad5314/volumes" Jul 12 22:39:09.096837 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.096801 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx"] Jul 12 22:39:09.097214 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.097130 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94ef04eb-b50e-4e8f-afef-b080e6ad5314" containerName="maas-api" Jul 12 22:39:09.097214 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.097140 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="94ef04eb-b50e-4e8f-afef-b080e6ad5314" containerName="maas-api" Jul 12 22:39:09.097214 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.097198 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="94ef04eb-b50e-4e8f-afef-b080e6ad5314" containerName="maas-api" Jul 12 22:39:09.101415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.101395 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.103996 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.103969 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-55d0a565-serving-cert\"" Jul 12 22:39:09.113143 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.113120 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx"] Jul 12 22:39:09.185264 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.185229 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-openshift-service-ca\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.185264 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.185262 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c6td\" (UniqueName: \"kubernetes.io/projected/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-kube-api-access-6c6td\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.185609 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.185297 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-maas-api-tls\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.286601 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.286562 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-maas-api-tls\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.286810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.286666 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-openshift-service-ca\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.286810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.286695 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c6td\" (UniqueName: \"kubernetes.io/projected/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-kube-api-access-6c6td\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.286810 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:39:09.286720 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-serving-cert: secret "maas-api-e2e-derive-55d0a565-serving-cert" not found Jul 12 22:39:09.286986 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:39:09.286815 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-maas-api-tls podName:f9c14cb8-d2ae-4530-82bf-ecd33dbfa530 nodeName:}" failed. No retries permitted until 2026-07-12 22:39:09.786789517 +0000 UTC m=+1731.655400333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-maas-api-tls") pod "maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" (UID: "f9c14cb8-d2ae-4530-82bf-ecd33dbfa530") : secret "maas-api-e2e-derive-55d0a565-serving-cert" not found Jul 12 22:39:09.287355 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.287335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-openshift-service-ca\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.295132 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.295109 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c6td\" (UniqueName: \"kubernetes.io/projected/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-kube-api-access-6c6td\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.791606 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.791564 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-maas-api-tls\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:09.794191 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:09.794154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-maas-api-tls\") pod \"maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:10.011895 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:10.011861 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:10.154263 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:10.154234 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx"] Jul 12 22:39:10.155877 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:39:10.155853 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9c14cb8_d2ae_4530_82bf_ecd33dbfa530.slice/crio-ad5fbc17e94e71cc1d6dc735132ba00e5aa5ae3285da3135c4175fbe6d7cc0ec WatchSource:0}: Error finding container ad5fbc17e94e71cc1d6dc735132ba00e5aa5ae3285da3135c4175fbe6d7cc0ec: Status 404 returned error can't find the container with id ad5fbc17e94e71cc1d6dc735132ba00e5aa5ae3285da3135c4175fbe6d7cc0ec Jul 12 22:39:11.100592 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:11.100551 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" event={"ID":"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530","Type":"ContainerStarted","Data":"f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c"} Jul 12 22:39:11.100772 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:11.100631 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" event={"ID":"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530","Type":"ContainerStarted","Data":"ad5fbc17e94e71cc1d6dc735132ba00e5aa5ae3285da3135c4175fbe6d7cc0ec"} Jul 12 22:39:11.100772 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:11.100654 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:39:11.119113 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:11.119055 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" podStartSLOduration=1.7909648310000001 podStartE2EDuration="2.119039687s" totalImagesPullingTime="328.074856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:39:11.11729457 +0000 UTC m=+1732.985905395" watchObservedRunningTime="2026-07-12 22:39:11.119039687 +0000 UTC m=+1732.987650512" Jul 12 22:39:17.111925 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:39:17.111893 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:40:14.000308 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.000236 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx"] Jul 12 22:40:14.001104 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.000557 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" podUID="f9c14cb8-d2ae-4530-82bf-ecd33dbfa530" containerName="maas-api" containerID="cri-o://f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c" gracePeriod=30 Jul 12 22:40:14.240290 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.240261 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:40:14.385456 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.385421 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-openshift-service-ca\") pod \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " Jul 12 22:40:14.385621 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.385474 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-maas-api-tls\") pod \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " Jul 12 22:40:14.385621 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.385546 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6c6td\" (UniqueName: \"kubernetes.io/projected/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-kube-api-access-6c6td\") pod \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\" (UID: \"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530\") " Jul 12 22:40:14.385813 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.385783 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f9c14cb8-d2ae-4530-82bf-ecd33dbfa530" (UID: "f9c14cb8-d2ae-4530-82bf-ecd33dbfa530"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:40:14.387796 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.387770 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f9c14cb8-d2ae-4530-82bf-ecd33dbfa530" (UID: "f9c14cb8-d2ae-4530-82bf-ecd33dbfa530"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:40:14.387894 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.387771 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-kube-api-access-6c6td" (OuterVolumeSpecName: "kube-api-access-6c6td") pod "f9c14cb8-d2ae-4530-82bf-ecd33dbfa530" (UID: "f9c14cb8-d2ae-4530-82bf-ecd33dbfa530"). InnerVolumeSpecName "kube-api-access-6c6td". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:40:14.422271 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.422244 2563 generic.go:356] "Generic (PLEG): container finished" podID="f9c14cb8-d2ae-4530-82bf-ecd33dbfa530" containerID="f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c" exitCode=0 Jul 12 22:40:14.422408 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.422292 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" Jul 12 22:40:14.422408 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.422320 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" event={"ID":"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530","Type":"ContainerDied","Data":"f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c"} Jul 12 22:40:14.422493 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.422407 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx" event={"ID":"f9c14cb8-d2ae-4530-82bf-ecd33dbfa530","Type":"ContainerDied","Data":"ad5fbc17e94e71cc1d6dc735132ba00e5aa5ae3285da3135c4175fbe6d7cc0ec"} Jul 12 22:40:14.422493 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.422429 2563 scope.go:117] "RemoveContainer" containerID="f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c" Jul 12 22:40:14.431708 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.431693 2563 scope.go:117] "RemoveContainer" containerID="f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c" Jul 12 22:40:14.431951 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:40:14.431931 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c\": container with ID starting with f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c not found: ID does not exist" containerID="f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c" Jul 12 22:40:14.432026 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.431964 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c"} err="failed to get container status \"f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c\": rpc error: code = NotFound desc = could not find container \"f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c\": container with ID starting with f5517afc303630852d3949f7baef56cf7f266135e6e6a3430e624aa101d53f9c not found: ID does not exist" Jul 12 22:40:14.443066 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.443033 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx"] Jul 12 22:40:14.446678 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.446655 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d0a565-ddc568cf5-fhhdx"] Jul 12 22:40:14.486834 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.486811 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:40:14.486834 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.486832 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:40:14.486959 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.486842 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6c6td\" (UniqueName: \"kubernetes.io/projected/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530-kube-api-access-6c6td\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:40:14.631514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:14.631485 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9c14cb8-d2ae-4530-82bf-ecd33dbfa530" path="/var/lib/kubelet/pods/f9c14cb8-d2ae-4530-82bf-ecd33dbfa530/volumes" Jul 12 22:40:18.727729 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:18.727626 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:40:18.745605 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:18.731971 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:40:18.745605 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:18.739945 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:40:18.745605 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:40:18.744735 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:42:08.753613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.753576 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b"] Jul 12 22:42:08.754212 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.754070 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9c14cb8-d2ae-4530-82bf-ecd33dbfa530" containerName="maas-api" Jul 12 22:42:08.754212 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.754084 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9c14cb8-d2ae-4530-82bf-ecd33dbfa530" containerName="maas-api" Jul 12 22:42:08.754212 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.754169 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9c14cb8-d2ae-4530-82bf-ecd33dbfa530" containerName="maas-api" Jul 12 22:42:08.757233 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.757206 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:08.759761 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.759735 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-10dc8c96-serving-cert\"" Jul 12 22:42:08.770554 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.770523 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b"] Jul 12 22:42:08.870459 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.870422 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5989409-1c79-44e4-9149-ef81da7d2036-openshift-service-ca\") pod \"maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:08.870650 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.870509 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlp9s\" (UniqueName: \"kubernetes.io/projected/c5989409-1c79-44e4-9149-ef81da7d2036-kube-api-access-nlp9s\") pod \"maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:08.870650 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.870541 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5989409-1c79-44e4-9149-ef81da7d2036-maas-api-tls\") pod \"maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:08.971895 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.971860 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5989409-1c79-44e4-9149-ef81da7d2036-openshift-service-ca\") pod \"maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:08.972082 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.971947 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlp9s\" (UniqueName: \"kubernetes.io/projected/c5989409-1c79-44e4-9149-ef81da7d2036-kube-api-access-nlp9s\") pod \"maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:08.972082 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.971991 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5989409-1c79-44e4-9149-ef81da7d2036-maas-api-tls\") pod \"maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:08.972655 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.972621 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5989409-1c79-44e4-9149-ef81da7d2036-openshift-service-ca\") pod \"maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:08.974471 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.974450 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5989409-1c79-44e4-9149-ef81da7d2036-maas-api-tls\") pod \"maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:08.980199 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:08.980174 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlp9s\" (UniqueName: \"kubernetes.io/projected/c5989409-1c79-44e4-9149-ef81da7d2036-kube-api-access-nlp9s\") pod \"maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:09.077321 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:09.077236 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:09.217681 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:09.217657 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b"] Jul 12 22:42:09.219789 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:42:09.219754 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5989409_1c79_44e4_9149_ef81da7d2036.slice/crio-e4fafd1bc5981c49f61fefc6a520a4587e624e7a55124992d6d946a6f8e8fef5 WatchSource:0}: Error finding container e4fafd1bc5981c49f61fefc6a520a4587e624e7a55124992d6d946a6f8e8fef5: Status 404 returned error can't find the container with id e4fafd1bc5981c49f61fefc6a520a4587e624e7a55124992d6d946a6f8e8fef5 Jul 12 22:42:09.221451 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:09.221433 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:42:10.008602 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:10.008517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" event={"ID":"c5989409-1c79-44e4-9149-ef81da7d2036","Type":"ContainerStarted","Data":"7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab"} Jul 12 22:42:10.008602 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:10.008569 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" event={"ID":"c5989409-1c79-44e4-9149-ef81da7d2036","Type":"ContainerStarted","Data":"e4fafd1bc5981c49f61fefc6a520a4587e624e7a55124992d6d946a6f8e8fef5"} Jul 12 22:42:10.008985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:10.008609 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:10.023987 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:10.023921 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" podStartSLOduration=1.4903692099999999 podStartE2EDuration="2.023902659s" totalImagesPullingTime="533.533449ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:42:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:42:10.023513362 +0000 UTC m=+1911.892124183" watchObservedRunningTime="2026-07-12 22:42:10.023902659 +0000 UTC m=+1911.892513485" Jul 12 22:42:16.019585 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:16.019557 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:31.516386 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.516336 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b"] Jul 12 22:42:31.516822 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.516593 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" podUID="c5989409-1c79-44e4-9149-ef81da7d2036" containerName="maas-api" containerID="cri-o://7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab" gracePeriod=30 Jul 12 22:42:31.771424 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.771334 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:31.878042 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.878005 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlp9s\" (UniqueName: \"kubernetes.io/projected/c5989409-1c79-44e4-9149-ef81da7d2036-kube-api-access-nlp9s\") pod \"c5989409-1c79-44e4-9149-ef81da7d2036\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " Jul 12 22:42:31.878042 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.878051 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5989409-1c79-44e4-9149-ef81da7d2036-openshift-service-ca\") pod \"c5989409-1c79-44e4-9149-ef81da7d2036\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " Jul 12 22:42:31.878300 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.878075 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5989409-1c79-44e4-9149-ef81da7d2036-maas-api-tls\") pod \"c5989409-1c79-44e4-9149-ef81da7d2036\" (UID: \"c5989409-1c79-44e4-9149-ef81da7d2036\") " Jul 12 22:42:31.878492 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.878467 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5989409-1c79-44e4-9149-ef81da7d2036-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5989409-1c79-44e4-9149-ef81da7d2036" (UID: "c5989409-1c79-44e4-9149-ef81da7d2036"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:42:31.880421 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.880390 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5989409-1c79-44e4-9149-ef81da7d2036-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5989409-1c79-44e4-9149-ef81da7d2036" (UID: "c5989409-1c79-44e4-9149-ef81da7d2036"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:42:31.880421 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.880392 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5989409-1c79-44e4-9149-ef81da7d2036-kube-api-access-nlp9s" (OuterVolumeSpecName: "kube-api-access-nlp9s") pod "c5989409-1c79-44e4-9149-ef81da7d2036" (UID: "c5989409-1c79-44e4-9149-ef81da7d2036"). InnerVolumeSpecName "kube-api-access-nlp9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:42:31.979164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.979122 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlp9s\" (UniqueName: \"kubernetes.io/projected/c5989409-1c79-44e4-9149-ef81da7d2036-kube-api-access-nlp9s\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:42:31.979164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.979156 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5989409-1c79-44e4-9149-ef81da7d2036-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:42:31.979164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:31.979170 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5989409-1c79-44e4-9149-ef81da7d2036-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:42:32.121871 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.121838 2563 generic.go:356] "Generic (PLEG): container finished" podID="c5989409-1c79-44e4-9149-ef81da7d2036" containerID="7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab" exitCode=0 Jul 12 22:42:32.122065 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.121891 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" Jul 12 22:42:32.122065 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.121926 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" event={"ID":"c5989409-1c79-44e4-9149-ef81da7d2036","Type":"ContainerDied","Data":"7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab"} Jul 12 22:42:32.122065 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.121985 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b" event={"ID":"c5989409-1c79-44e4-9149-ef81da7d2036","Type":"ContainerDied","Data":"e4fafd1bc5981c49f61fefc6a520a4587e624e7a55124992d6d946a6f8e8fef5"} Jul 12 22:42:32.122065 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.122014 2563 scope.go:117] "RemoveContainer" containerID="7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab" Jul 12 22:42:32.130942 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.130925 2563 scope.go:117] "RemoveContainer" containerID="7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab" Jul 12 22:42:32.131225 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:42:32.131207 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab\": container with ID starting with 7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab not found: ID does not exist" containerID="7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab" Jul 12 22:42:32.131280 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.131234 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab"} err="failed to get container status \"7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab\": rpc error: code = NotFound desc = could not find container \"7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab\": container with ID starting with 7cf90a93b3b7f92ec339b51995eb29fb521d389f6d71383dab60c82a6e7ea3ab not found: ID does not exist" Jul 12 22:42:32.144904 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.144876 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b"] Jul 12 22:42:32.149060 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.149035 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10dc8c96-54bb79fd86-5xz2b"] Jul 12 22:42:32.630914 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:42:32.630882 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5989409-1c79-44e4-9149-ef81da7d2036" path="/var/lib/kubelet/pods/c5989409-1c79-44e4-9149-ef81da7d2036/volumes" Jul 12 22:43:09.063608 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.063571 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr"] Jul 12 22:43:09.064228 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.063962 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5989409-1c79-44e4-9149-ef81da7d2036" containerName="maas-api" Jul 12 22:43:09.064228 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.063975 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5989409-1c79-44e4-9149-ef81da7d2036" containerName="maas-api" Jul 12 22:43:09.064228 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.064050 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5989409-1c79-44e4-9149-ef81da7d2036" containerName="maas-api" Jul 12 22:43:09.066995 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.066977 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.069251 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.069231 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3f30527e-serving-cert\"" Jul 12 22:43:09.075863 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.075837 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr"] Jul 12 22:43:09.202580 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.202544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v22g\" (UniqueName: \"kubernetes.io/projected/efbe6ee2-9769-47ff-a38d-cf48ac520927-kube-api-access-9v22g\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.202758 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.202584 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbe6ee2-9769-47ff-a38d-cf48ac520927-openshift-service-ca\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.202758 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.202684 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efbe6ee2-9769-47ff-a38d-cf48ac520927-maas-api-tls\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.303957 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.303922 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efbe6ee2-9769-47ff-a38d-cf48ac520927-maas-api-tls\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.304139 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.304013 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v22g\" (UniqueName: \"kubernetes.io/projected/efbe6ee2-9769-47ff-a38d-cf48ac520927-kube-api-access-9v22g\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.304139 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.304044 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbe6ee2-9769-47ff-a38d-cf48ac520927-openshift-service-ca\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.304139 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:43:09.304056 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-serving-cert: secret "maas-api-e2e-mt-3f30527e-serving-cert" not found Jul 12 22:43:09.304139 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:43:09.304128 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efbe6ee2-9769-47ff-a38d-cf48ac520927-maas-api-tls podName:efbe6ee2-9769-47ff-a38d-cf48ac520927 nodeName:}" failed. No retries permitted until 2026-07-12 22:43:09.804112534 +0000 UTC m=+1971.672723336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/efbe6ee2-9769-47ff-a38d-cf48ac520927-maas-api-tls") pod "maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" (UID: "efbe6ee2-9769-47ff-a38d-cf48ac520927") : secret "maas-api-e2e-mt-3f30527e-serving-cert" not found Jul 12 22:43:09.304818 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.304798 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbe6ee2-9769-47ff-a38d-cf48ac520927-openshift-service-ca\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.312682 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.312663 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v22g\" (UniqueName: \"kubernetes.io/projected/efbe6ee2-9769-47ff-a38d-cf48ac520927-kube-api-access-9v22g\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.809274 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.809176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efbe6ee2-9769-47ff-a38d-cf48ac520927-maas-api-tls\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.811810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.811784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efbe6ee2-9769-47ff-a38d-cf48ac520927-maas-api-tls\") pod \"maas-api-e2e-mt-3f30527e-5486b694cc-vprnr\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:09.978897 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:09.978866 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:10.118294 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:10.118271 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr"] Jul 12 22:43:10.119819 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:43:10.119788 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefbe6ee2_9769_47ff_a38d_cf48ac520927.slice/crio-191552a7303079ab182150962e79399fa49a105d6a3f9b649e2261cbdcfeed40 WatchSource:0}: Error finding container 191552a7303079ab182150962e79399fa49a105d6a3f9b649e2261cbdcfeed40: Status 404 returned error can't find the container with id 191552a7303079ab182150962e79399fa49a105d6a3f9b649e2261cbdcfeed40 Jul 12 22:43:10.309911 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:10.309876 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" event={"ID":"efbe6ee2-9769-47ff-a38d-cf48ac520927","Type":"ContainerStarted","Data":"191552a7303079ab182150962e79399fa49a105d6a3f9b649e2261cbdcfeed40"} Jul 12 22:43:11.316908 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:11.316873 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" event={"ID":"efbe6ee2-9769-47ff-a38d-cf48ac520927","Type":"ContainerStarted","Data":"eb83ed99b47f37868f17f92e1508c44de06cb28ffbda7251ce804c991468d247"} Jul 12 22:43:11.317405 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:11.317044 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:43:11.332989 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:11.332945 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" podStartSLOduration=1.974614677 podStartE2EDuration="2.332932759s" totalImagesPullingTime="358.318082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:43:11.331290673 +0000 UTC m=+1973.199901496" watchObservedRunningTime="2026-07-12 22:43:11.332932759 +0000 UTC m=+1973.201543582" Jul 12 22:43:17.326950 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:43:17.326918 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:44:07.439140 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.439101 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr"] Jul 12 22:44:07.439667 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.439302 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" podUID="efbe6ee2-9769-47ff-a38d-cf48ac520927" containerName="maas-api" containerID="cri-o://eb83ed99b47f37868f17f92e1508c44de06cb28ffbda7251ce804c991468d247" gracePeriod=30 Jul 12 22:44:07.589069 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.589037 2563 generic.go:356] "Generic (PLEG): container finished" podID="efbe6ee2-9769-47ff-a38d-cf48ac520927" containerID="eb83ed99b47f37868f17f92e1508c44de06cb28ffbda7251ce804c991468d247" exitCode=0 Jul 12 22:44:07.589237 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.589114 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" event={"ID":"efbe6ee2-9769-47ff-a38d-cf48ac520927","Type":"ContainerDied","Data":"eb83ed99b47f37868f17f92e1508c44de06cb28ffbda7251ce804c991468d247"} Jul 12 22:44:07.684236 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.684210 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:44:07.779301 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.779220 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v22g\" (UniqueName: \"kubernetes.io/projected/efbe6ee2-9769-47ff-a38d-cf48ac520927-kube-api-access-9v22g\") pod \"efbe6ee2-9769-47ff-a38d-cf48ac520927\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " Jul 12 22:44:07.779301 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.779269 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbe6ee2-9769-47ff-a38d-cf48ac520927-openshift-service-ca\") pod \"efbe6ee2-9769-47ff-a38d-cf48ac520927\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " Jul 12 22:44:07.779301 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.779295 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efbe6ee2-9769-47ff-a38d-cf48ac520927-maas-api-tls\") pod \"efbe6ee2-9769-47ff-a38d-cf48ac520927\" (UID: \"efbe6ee2-9769-47ff-a38d-cf48ac520927\") " Jul 12 22:44:07.779735 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.779706 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efbe6ee2-9769-47ff-a38d-cf48ac520927-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efbe6ee2-9769-47ff-a38d-cf48ac520927" (UID: "efbe6ee2-9769-47ff-a38d-cf48ac520927"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:44:07.781441 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.781419 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efbe6ee2-9769-47ff-a38d-cf48ac520927-kube-api-access-9v22g" (OuterVolumeSpecName: "kube-api-access-9v22g") pod "efbe6ee2-9769-47ff-a38d-cf48ac520927" (UID: "efbe6ee2-9769-47ff-a38d-cf48ac520927"). InnerVolumeSpecName "kube-api-access-9v22g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:44:07.781620 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.781597 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efbe6ee2-9769-47ff-a38d-cf48ac520927-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efbe6ee2-9769-47ff-a38d-cf48ac520927" (UID: "efbe6ee2-9769-47ff-a38d-cf48ac520927"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:44:07.880470 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.880437 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v22g\" (UniqueName: \"kubernetes.io/projected/efbe6ee2-9769-47ff-a38d-cf48ac520927-kube-api-access-9v22g\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:44:07.880470 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.880462 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efbe6ee2-9769-47ff-a38d-cf48ac520927-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:44:07.880470 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:07.880472 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efbe6ee2-9769-47ff-a38d-cf48ac520927-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:44:08.595643 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:08.595616 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" Jul 12 22:44:08.596088 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:08.595614 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr" event={"ID":"efbe6ee2-9769-47ff-a38d-cf48ac520927","Type":"ContainerDied","Data":"191552a7303079ab182150962e79399fa49a105d6a3f9b649e2261cbdcfeed40"} Jul 12 22:44:08.596088 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:08.595754 2563 scope.go:117] "RemoveContainer" containerID="eb83ed99b47f37868f17f92e1508c44de06cb28ffbda7251ce804c991468d247" Jul 12 22:44:08.616656 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:08.616633 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr"] Jul 12 22:44:08.618534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:08.618516 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3f30527e-5486b694cc-vprnr"] Jul 12 22:44:08.631772 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:44:08.631750 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efbe6ee2-9769-47ff-a38d-cf48ac520927" path="/var/lib/kubelet/pods/efbe6ee2-9769-47ff-a38d-cf48ac520927/volumes" Jul 12 22:45:18.765392 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:45:18.765249 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:45:18.771765 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:45:18.771743 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:45:18.772144 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:45:18.772125 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:45:18.777964 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:45:18.777949 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:46:02.044255 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.044219 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq"] Jul 12 22:46:02.044872 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.044746 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbe6ee2-9769-47ff-a38d-cf48ac520927" containerName="maas-api" Jul 12 22:46:02.044872 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.044766 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbe6ee2-9769-47ff-a38d-cf48ac520927" containerName="maas-api" Jul 12 22:46:02.044998 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.044886 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="efbe6ee2-9769-47ff-a38d-cf48ac520927" containerName="maas-api" Jul 12 22:46:02.047595 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.047572 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.050214 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.050187 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-30d729-serving-cert\"" Jul 12 22:46:02.059842 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.059819 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq"] Jul 12 22:46:02.149669 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.149633 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.149830 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.149689 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7mkc\" (UniqueName: \"kubernetes.io/projected/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-kube-api-access-l7mkc\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.149830 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.149740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-maas-api-tls\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.250175 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.250143 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.250403 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.250194 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7mkc\" (UniqueName: \"kubernetes.io/projected/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-kube-api-access-l7mkc\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.250403 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.250225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-maas-api-tls\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.250403 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:46:02.250394 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-serving-cert: secret "maas-api-e2e-inf-a-30d729-serving-cert" not found Jul 12 22:46:02.250605 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:46:02.250474 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-maas-api-tls podName:0b1ef227-8d0c-4c60-8b1b-928593bfe41b nodeName:}" failed. No retries permitted until 2026-07-12 22:46:02.750451391 +0000 UTC m=+2144.619062193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-maas-api-tls") pod "maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" (UID: "0b1ef227-8d0c-4c60-8b1b-928593bfe41b") : secret "maas-api-e2e-inf-a-30d729-serving-cert" not found Jul 12 22:46:02.250798 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.250777 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.264203 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.264174 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7mkc\" (UniqueName: \"kubernetes.io/projected/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-kube-api-access-l7mkc\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.754671 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.754641 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-maas-api-tls\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.757180 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.757160 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-maas-api-tls\") pod \"maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:02.958512 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:02.958474 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:03.103062 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:03.101161 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq"] Jul 12 22:46:03.162161 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:03.162127 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" event={"ID":"0b1ef227-8d0c-4c60-8b1b-928593bfe41b","Type":"ContainerStarted","Data":"29fdf3c812c44443baae73acafd74166a9e83aaec5ee65b9fdf0c5dc97769cfb"} Jul 12 22:46:04.168312 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:04.168272 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" event={"ID":"0b1ef227-8d0c-4c60-8b1b-928593bfe41b","Type":"ContainerStarted","Data":"aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d"} Jul 12 22:46:04.168733 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:04.168415 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:04.186029 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:04.185977 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" podStartSLOduration=1.712641933 podStartE2EDuration="2.185961944s" totalImagesPullingTime="473.320011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:46:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:46:04.183350925 +0000 UTC m=+2146.051961749" watchObservedRunningTime="2026-07-12 22:46:04.185961944 +0000 UTC m=+2146.054572824" Jul 12 22:46:10.020034 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.019945 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc"] Jul 12 22:46:10.031772 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.031743 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.035725 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.035696 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-58d7a8-serving-cert\"" Jul 12 22:46:10.036016 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.035994 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc"] Jul 12 22:46:10.116652 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.116619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh9zk\" (UniqueName: \"kubernetes.io/projected/962ec347-f0f6-4224-91c5-1d405aba047f-kube-api-access-lh9zk\") pod \"maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.116796 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.116659 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962ec347-f0f6-4224-91c5-1d405aba047f-maas-api-tls\") pod \"maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.116839 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.116796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962ec347-f0f6-4224-91c5-1d405aba047f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.178304 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.178276 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:46:10.217273 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.217241 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh9zk\" (UniqueName: \"kubernetes.io/projected/962ec347-f0f6-4224-91c5-1d405aba047f-kube-api-access-lh9zk\") pod \"maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.217273 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.217276 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962ec347-f0f6-4224-91c5-1d405aba047f-maas-api-tls\") pod \"maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.217538 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.217346 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962ec347-f0f6-4224-91c5-1d405aba047f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.218160 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.218134 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962ec347-f0f6-4224-91c5-1d405aba047f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.220176 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.220155 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962ec347-f0f6-4224-91c5-1d405aba047f-maas-api-tls\") pod \"maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.225328 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.225298 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh9zk\" (UniqueName: \"kubernetes.io/projected/962ec347-f0f6-4224-91c5-1d405aba047f-kube-api-access-lh9zk\") pod \"maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.345713 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.345685 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:10.484818 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:10.484788 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc"] Jul 12 22:46:10.487676 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:46:10.487643 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962ec347_f0f6_4224_91c5_1d405aba047f.slice/crio-a9437735c868c93b08839346c6021e7247c8da054471934b9790723e69086736 WatchSource:0}: Error finding container a9437735c868c93b08839346c6021e7247c8da054471934b9790723e69086736: Status 404 returned error can't find the container with id a9437735c868c93b08839346c6021e7247c8da054471934b9790723e69086736 Jul 12 22:46:11.204649 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:11.204609 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" event={"ID":"962ec347-f0f6-4224-91c5-1d405aba047f","Type":"ContainerStarted","Data":"3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc"} Jul 12 22:46:11.205112 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:11.204701 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" event={"ID":"962ec347-f0f6-4224-91c5-1d405aba047f","Type":"ContainerStarted","Data":"a9437735c868c93b08839346c6021e7247c8da054471934b9790723e69086736"} Jul 12 22:46:11.205112 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:11.204726 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:46:11.221779 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:11.221732 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" podStartSLOduration=0.887006465 podStartE2EDuration="1.221715541s" totalImagesPullingTime="334.709076ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:46:11.219521701 +0000 UTC m=+2153.088132525" watchObservedRunningTime="2026-07-12 22:46:11.221715541 +0000 UTC m=+2153.090326396" Jul 12 22:46:17.214569 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:46:17.214542 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:47:39.804722 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:39.804631 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq"] Jul 12 22:47:39.805162 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:39.804854 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" podUID="0b1ef227-8d0c-4c60-8b1b-928593bfe41b" containerName="maas-api" containerID="cri-o://aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d" gracePeriod=30 Jul 12 22:47:40.241670 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.241646 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:47:40.376062 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.376020 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7mkc\" (UniqueName: \"kubernetes.io/projected/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-kube-api-access-l7mkc\") pod \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " Jul 12 22:47:40.376239 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.376085 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-maas-api-tls\") pod \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " Jul 12 22:47:40.376239 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.376114 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-openshift-service-ca\") pod \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\" (UID: \"0b1ef227-8d0c-4c60-8b1b-928593bfe41b\") " Jul 12 22:47:40.376547 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.376521 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0b1ef227-8d0c-4c60-8b1b-928593bfe41b" (UID: "0b1ef227-8d0c-4c60-8b1b-928593bfe41b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:47:40.378243 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.378215 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-kube-api-access-l7mkc" (OuterVolumeSpecName: "kube-api-access-l7mkc") pod "0b1ef227-8d0c-4c60-8b1b-928593bfe41b" (UID: "0b1ef227-8d0c-4c60-8b1b-928593bfe41b"). InnerVolumeSpecName "kube-api-access-l7mkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:47:40.378243 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.378227 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0b1ef227-8d0c-4c60-8b1b-928593bfe41b" (UID: "0b1ef227-8d0c-4c60-8b1b-928593bfe41b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:47:40.477191 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.477164 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7mkc\" (UniqueName: \"kubernetes.io/projected/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-kube-api-access-l7mkc\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:47:40.477191 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.477188 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:47:40.477191 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.477197 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b1ef227-8d0c-4c60-8b1b-928593bfe41b-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:47:40.653306 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.653273 2563 generic.go:356] "Generic (PLEG): container finished" podID="0b1ef227-8d0c-4c60-8b1b-928593bfe41b" containerID="aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d" exitCode=0 Jul 12 22:47:40.653468 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.653324 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" Jul 12 22:47:40.653468 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.653349 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" event={"ID":"0b1ef227-8d0c-4c60-8b1b-928593bfe41b","Type":"ContainerDied","Data":"aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d"} Jul 12 22:47:40.653468 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.653417 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" event={"ID":"0b1ef227-8d0c-4c60-8b1b-928593bfe41b","Type":"ContainerDied","Data":"29fdf3c812c44443baae73acafd74166a9e83aaec5ee65b9fdf0c5dc97769cfb"} Jul 12 22:47:40.653468 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.653439 2563 scope.go:117] "RemoveContainer" containerID="aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d" Jul 12 22:47:40.662212 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.662194 2563 scope.go:117] "RemoveContainer" containerID="aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d" Jul 12 22:47:40.662549 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:47:40.662531 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d\": container with ID starting with aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d not found: ID does not exist" containerID="aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d" Jul 12 22:47:40.662607 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.662559 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d"} err="failed to get container status \"aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d\": rpc error: code = NotFound desc = could not find container \"aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d\": container with ID starting with aa69592112c0fb28837a16c05c19b7cfce05dd0410db3dfdc7dc0ae1fda1a59d not found: ID does not exist" Jul 12 22:47:40.669642 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.669621 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq"] Jul 12 22:47:40.673313 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:40.673290 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq"] Jul 12 22:47:42.631001 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:42.630969 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b1ef227-8d0c-4c60-8b1b-928593bfe41b" path="/var/lib/kubelet/pods/0b1ef227-8d0c-4c60-8b1b-928593bfe41b/volumes" Jul 12 22:47:43.174264 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:47:43.174207 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-30d729-5b8c6466cf-hjcjq" podUID="0b1ef227-8d0c-4c60-8b1b-928593bfe41b" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.50:8443/health\": context deadline exceeded" Jul 12 22:48:05.880018 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:05.879983 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc"] Jul 12 22:48:05.880515 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:05.880261 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" podUID="962ec347-f0f6-4224-91c5-1d405aba047f" containerName="maas-api" containerID="cri-o://3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc" gracePeriod=30 Jul 12 22:48:07.123746 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.123717 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:48:07.210088 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.210063 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962ec347-f0f6-4224-91c5-1d405aba047f-maas-api-tls\") pod \"962ec347-f0f6-4224-91c5-1d405aba047f\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " Jul 12 22:48:07.210244 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.210133 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh9zk\" (UniqueName: \"kubernetes.io/projected/962ec347-f0f6-4224-91c5-1d405aba047f-kube-api-access-lh9zk\") pod \"962ec347-f0f6-4224-91c5-1d405aba047f\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " Jul 12 22:48:07.210244 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.210168 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962ec347-f0f6-4224-91c5-1d405aba047f-openshift-service-ca\") pod \"962ec347-f0f6-4224-91c5-1d405aba047f\" (UID: \"962ec347-f0f6-4224-91c5-1d405aba047f\") " Jul 12 22:48:07.210618 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.210594 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/962ec347-f0f6-4224-91c5-1d405aba047f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "962ec347-f0f6-4224-91c5-1d405aba047f" (UID: "962ec347-f0f6-4224-91c5-1d405aba047f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:48:07.212185 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.212156 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/962ec347-f0f6-4224-91c5-1d405aba047f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "962ec347-f0f6-4224-91c5-1d405aba047f" (UID: "962ec347-f0f6-4224-91c5-1d405aba047f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:48:07.212388 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.212352 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/962ec347-f0f6-4224-91c5-1d405aba047f-kube-api-access-lh9zk" (OuterVolumeSpecName: "kube-api-access-lh9zk") pod "962ec347-f0f6-4224-91c5-1d405aba047f" (UID: "962ec347-f0f6-4224-91c5-1d405aba047f"). InnerVolumeSpecName "kube-api-access-lh9zk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:48:07.311427 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.311341 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh9zk\" (UniqueName: \"kubernetes.io/projected/962ec347-f0f6-4224-91c5-1d405aba047f-kube-api-access-lh9zk\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:48:07.311427 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.311387 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/962ec347-f0f6-4224-91c5-1d405aba047f-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:48:07.311427 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.311398 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/962ec347-f0f6-4224-91c5-1d405aba047f-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:48:07.786183 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.786149 2563 generic.go:356] "Generic (PLEG): container finished" podID="962ec347-f0f6-4224-91c5-1d405aba047f" containerID="3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc" exitCode=0 Jul 12 22:48:07.786183 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.786186 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" Jul 12 22:48:07.786401 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.786238 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" event={"ID":"962ec347-f0f6-4224-91c5-1d405aba047f","Type":"ContainerDied","Data":"3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc"} Jul 12 22:48:07.786401 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.786283 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc" event={"ID":"962ec347-f0f6-4224-91c5-1d405aba047f","Type":"ContainerDied","Data":"a9437735c868c93b08839346c6021e7247c8da054471934b9790723e69086736"} Jul 12 22:48:07.786401 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.786304 2563 scope.go:117] "RemoveContainer" containerID="3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc" Jul 12 22:48:07.795515 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.795480 2563 scope.go:117] "RemoveContainer" containerID="3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc" Jul 12 22:48:07.795782 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:48:07.795759 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc\": container with ID starting with 3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc not found: ID does not exist" containerID="3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc" Jul 12 22:48:07.795824 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.795790 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc"} err="failed to get container status \"3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc\": rpc error: code = NotFound desc = could not find container \"3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc\": container with ID starting with 3156c9be3eb48e5e604c7e65bcfef2b639cb7732ef72c9a8d7cf9f8a462cbafc not found: ID does not exist" Jul 12 22:48:07.806724 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.806699 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc"] Jul 12 22:48:07.812449 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:07.812428 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-58d7a8-6869c7dcc4-7btgc"] Jul 12 22:48:08.630938 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:08.630904 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="962ec347-f0f6-4224-91c5-1d405aba047f" path="/var/lib/kubelet/pods/962ec347-f0f6-4224-91c5-1d405aba047f/volumes" Jul 12 22:48:24.898231 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.898197 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h"] Jul 12 22:48:24.898928 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.898538 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962ec347-f0f6-4224-91c5-1d405aba047f" containerName="maas-api" Jul 12 22:48:24.898928 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.898551 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="962ec347-f0f6-4224-91c5-1d405aba047f" containerName="maas-api" Jul 12 22:48:24.898928 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.898567 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b1ef227-8d0c-4c60-8b1b-928593bfe41b" containerName="maas-api" Jul 12 22:48:24.898928 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.898572 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b1ef227-8d0c-4c60-8b1b-928593bfe41b" containerName="maas-api" Jul 12 22:48:24.898928 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.898639 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="962ec347-f0f6-4224-91c5-1d405aba047f" containerName="maas-api" Jul 12 22:48:24.898928 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.898649 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b1ef227-8d0c-4c60-8b1b-928593bfe41b" containerName="maas-api" Jul 12 22:48:24.901514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.901497 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:24.903676 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.903661 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-8a9e41-serving-cert\"" Jul 12 22:48:24.911463 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.911435 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h"] Jul 12 22:48:24.962906 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.962872 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjlnp\" (UniqueName: \"kubernetes.io/projected/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-kube-api-access-tjlnp\") pod \"maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:24.963092 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.962929 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-openshift-service-ca\") pod \"maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:24.963092 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:24.962951 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-maas-api-tls\") pod \"maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:25.063740 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.063709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-openshift-service-ca\") pod \"maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:25.063740 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.063744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-maas-api-tls\") pod \"maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:25.063946 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.063842 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjlnp\" (UniqueName: \"kubernetes.io/projected/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-kube-api-access-tjlnp\") pod \"maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:25.064350 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.064321 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-openshift-service-ca\") pod \"maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:25.066583 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.066563 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-maas-api-tls\") pod \"maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:25.072024 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.072001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjlnp\" (UniqueName: \"kubernetes.io/projected/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-kube-api-access-tjlnp\") pod \"maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:25.212703 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.212609 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:25.348079 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.348051 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h"] Jul 12 22:48:25.352898 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.352875 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:48:25.878100 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.878064 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" event={"ID":"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7","Type":"ContainerStarted","Data":"4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9"} Jul 12 22:48:25.878304 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.878126 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" event={"ID":"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7","Type":"ContainerStarted","Data":"08a89dcb2e7123399abc6a8c22e1be515585dc92277a30a5eef59873fab2dca9"} Jul 12 22:48:25.878304 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.878153 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:25.898190 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:25.898139 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" podStartSLOduration=1.526420023 podStartE2EDuration="1.898125644s" totalImagesPullingTime="371.705621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:48:25.895574996 +0000 UTC m=+2287.764185823" watchObservedRunningTime="2026-07-12 22:48:25.898125644 +0000 UTC m=+2287.766736467" Jul 12 22:48:31.888889 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:31.888864 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:33.106731 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.106700 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9"] Jul 12 22:48:33.110484 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.110460 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.112847 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.112824 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-189e5d-serving-cert\"" Jul 12 22:48:33.121532 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.121506 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9"] Jul 12 22:48:33.236482 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.236446 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-maas-api-tls\") pod \"maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.236482 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.236479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-openshift-service-ca\") pod \"maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.236736 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.236585 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsrc8\" (UniqueName: \"kubernetes.io/projected/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-kube-api-access-bsrc8\") pod \"maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.337663 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.337629 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsrc8\" (UniqueName: \"kubernetes.io/projected/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-kube-api-access-bsrc8\") pod \"maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.337842 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.337706 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-maas-api-tls\") pod \"maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.337842 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.337732 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-openshift-service-ca\") pod \"maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.338463 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.338440 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-openshift-service-ca\") pod \"maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.340280 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.340255 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-maas-api-tls\") pod \"maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.347632 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.347612 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsrc8\" (UniqueName: \"kubernetes.io/projected/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-kube-api-access-bsrc8\") pod \"maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.422587 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.422510 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:33.557737 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.557713 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9"] Jul 12 22:48:33.560010 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:48:33.559980 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c5c5681_d3af_4947_b1ad_8e2b2aa6a281.slice/crio-c05d7e3f016a59b667717e089f5cefe2fe2d4c80bcc6057785e81a75208788a0 WatchSource:0}: Error finding container c05d7e3f016a59b667717e089f5cefe2fe2d4c80bcc6057785e81a75208788a0: Status 404 returned error can't find the container with id c05d7e3f016a59b667717e089f5cefe2fe2d4c80bcc6057785e81a75208788a0 Jul 12 22:48:33.919826 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:33.919783 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" event={"ID":"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281","Type":"ContainerStarted","Data":"c05d7e3f016a59b667717e089f5cefe2fe2d4c80bcc6057785e81a75208788a0"} Jul 12 22:48:34.925689 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:34.925654 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" event={"ID":"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281","Type":"ContainerStarted","Data":"e46057add91f777b5725731b563b7e4b8f97270638bba85980ab8953d012036d"} Jul 12 22:48:34.926103 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:34.925784 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:34.944060 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:34.944015 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" podStartSLOduration=1.581703098 podStartE2EDuration="1.944003807s" totalImagesPullingTime="362.300709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:48:34.941360721 +0000 UTC m=+2296.809971544" watchObservedRunningTime="2026-07-12 22:48:34.944003807 +0000 UTC m=+2296.812614631" Jul 12 22:48:40.941651 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:40.941623 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:48:56.307509 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.307472 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h"] Jul 12 22:48:56.308084 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.307716 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" podUID="a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7" containerName="maas-api" containerID="cri-o://4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9" gracePeriod=30 Jul 12 22:48:56.551746 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.551720 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:56.641381 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.641349 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-openshift-service-ca\") pod \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " Jul 12 22:48:56.641559 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.641425 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-maas-api-tls\") pod \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " Jul 12 22:48:56.641559 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.641532 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjlnp\" (UniqueName: \"kubernetes.io/projected/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-kube-api-access-tjlnp\") pod \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\" (UID: \"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7\") " Jul 12 22:48:56.641796 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.641775 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7" (UID: "a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:48:56.643611 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.643591 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7" (UID: "a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:48:56.643711 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.643669 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-kube-api-access-tjlnp" (OuterVolumeSpecName: "kube-api-access-tjlnp") pod "a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7" (UID: "a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7"). InnerVolumeSpecName "kube-api-access-tjlnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:48:56.742598 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.742570 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:48:56.742598 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.742595 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:48:56.742598 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:56.742604 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjlnp\" (UniqueName: \"kubernetes.io/projected/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7-kube-api-access-tjlnp\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:48:57.042595 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:57.042502 2563 generic.go:356] "Generic (PLEG): container finished" podID="a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7" containerID="4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9" exitCode=0 Jul 12 22:48:57.042595 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:57.042552 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" Jul 12 22:48:57.042805 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:57.042583 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" event={"ID":"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7","Type":"ContainerDied","Data":"4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9"} Jul 12 22:48:57.042805 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:57.042642 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h" event={"ID":"a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7","Type":"ContainerDied","Data":"08a89dcb2e7123399abc6a8c22e1be515585dc92277a30a5eef59873fab2dca9"} Jul 12 22:48:57.042805 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:57.042666 2563 scope.go:117] "RemoveContainer" containerID="4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9" Jul 12 22:48:57.051851 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:57.051833 2563 scope.go:117] "RemoveContainer" containerID="4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9" Jul 12 22:48:57.052104 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:48:57.052087 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9\": container with ID starting with 4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9 not found: ID does not exist" containerID="4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9" Jul 12 22:48:57.052173 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:57.052112 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9"} err="failed to get container status \"4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9\": rpc error: code = NotFound desc = could not find container \"4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9\": container with ID starting with 4110ec3d517713cd42607e7fc13f101d3ab9c9732100f371d40265f9bca2d5f9 not found: ID does not exist" Jul 12 22:48:57.063972 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:57.063948 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h"] Jul 12 22:48:57.069452 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:57.069430 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8a9e41-d86b9985f-s7n7h"] Jul 12 22:48:58.631186 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:48:58.631145 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7" path="/var/lib/kubelet/pods/a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7/volumes" Jul 12 22:49:22.048770 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.048735 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9"] Jul 12 22:49:22.049238 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.048982 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" podUID="1c5c5681-d3af-4947-b1ad-8e2b2aa6a281" containerName="maas-api" containerID="cri-o://e46057add91f777b5725731b563b7e4b8f97270638bba85980ab8953d012036d" gracePeriod=30 Jul 12 22:49:22.176292 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.176258 2563 generic.go:356] "Generic (PLEG): container finished" podID="1c5c5681-d3af-4947-b1ad-8e2b2aa6a281" containerID="e46057add91f777b5725731b563b7e4b8f97270638bba85980ab8953d012036d" exitCode=0 Jul 12 22:49:22.176436 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.176324 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" event={"ID":"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281","Type":"ContainerDied","Data":"e46057add91f777b5725731b563b7e4b8f97270638bba85980ab8953d012036d"} Jul 12 22:49:22.296564 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.296533 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:49:22.469056 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.469026 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsrc8\" (UniqueName: \"kubernetes.io/projected/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-kube-api-access-bsrc8\") pod \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " Jul 12 22:49:22.469245 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.469079 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-maas-api-tls\") pod \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " Jul 12 22:49:22.469245 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.469144 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-openshift-service-ca\") pod \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\" (UID: \"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281\") " Jul 12 22:49:22.469641 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.469600 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c5c5681-d3af-4947-b1ad-8e2b2aa6a281" (UID: "1c5c5681-d3af-4947-b1ad-8e2b2aa6a281"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:49:22.471397 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.471344 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c5c5681-d3af-4947-b1ad-8e2b2aa6a281" (UID: "1c5c5681-d3af-4947-b1ad-8e2b2aa6a281"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:49:22.471506 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.471315 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-kube-api-access-bsrc8" (OuterVolumeSpecName: "kube-api-access-bsrc8") pod "1c5c5681-d3af-4947-b1ad-8e2b2aa6a281" (UID: "1c5c5681-d3af-4947-b1ad-8e2b2aa6a281"). InnerVolumeSpecName "kube-api-access-bsrc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:49:22.570517 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.570484 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsrc8\" (UniqueName: \"kubernetes.io/projected/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-kube-api-access-bsrc8\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:49:22.570517 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.570516 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:49:22.570716 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:22.570526 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:49:23.182425 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:23.182391 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" Jul 12 22:49:23.182425 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:23.182404 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9" event={"ID":"1c5c5681-d3af-4947-b1ad-8e2b2aa6a281","Type":"ContainerDied","Data":"c05d7e3f016a59b667717e089f5cefe2fe2d4c80bcc6057785e81a75208788a0"} Jul 12 22:49:23.182932 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:23.182481 2563 scope.go:117] "RemoveContainer" containerID="e46057add91f777b5725731b563b7e4b8f97270638bba85980ab8953d012036d" Jul 12 22:49:23.199984 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:23.199958 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9"] Jul 12 22:49:23.203299 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:23.203277 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-189e5d-7bb75f48cb-mtdz9"] Jul 12 22:49:24.630308 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:24.630273 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c5c5681-d3af-4947-b1ad-8e2b2aa6a281" path="/var/lib/kubelet/pods/1c5c5681-d3af-4947-b1ad-8e2b2aa6a281/volumes" Jul 12 22:49:41.311784 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.311737 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9"] Jul 12 22:49:41.312167 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.312116 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7" containerName="maas-api" Jul 12 22:49:41.312167 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.312129 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7" containerName="maas-api" Jul 12 22:49:41.312251 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.312179 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c5c5681-d3af-4947-b1ad-8e2b2aa6a281" containerName="maas-api" Jul 12 22:49:41.312251 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.312190 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c5c5681-d3af-4947-b1ad-8e2b2aa6a281" containerName="maas-api" Jul 12 22:49:41.312361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.312288 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a61e7c51-2a00-4aea-b221-4d2ce0d0d2f7" containerName="maas-api" Jul 12 22:49:41.312361 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.312303 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c5c5681-d3af-4947-b1ad-8e2b2aa6a281" containerName="maas-api" Jul 12 22:49:41.315790 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.315733 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.318355 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.318327 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-e321a4-serving-cert\"" Jul 12 22:49:41.324820 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.324612 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9"] Jul 12 22:49:41.327401 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.327354 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4d24a83-8d62-4cf2-877f-190331f4ed77-maas-api-tls\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.327581 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.327562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4d24a83-8d62-4cf2-877f-190331f4ed77-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.327721 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.327707 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw2qv\" (UniqueName: \"kubernetes.io/projected/b4d24a83-8d62-4cf2-877f-190331f4ed77-kube-api-access-kw2qv\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.428731 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.428692 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4d24a83-8d62-4cf2-877f-190331f4ed77-maas-api-tls\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.428731 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.428731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4d24a83-8d62-4cf2-877f-190331f4ed77-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.428985 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.428786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw2qv\" (UniqueName: \"kubernetes.io/projected/b4d24a83-8d62-4cf2-877f-190331f4ed77-kube-api-access-kw2qv\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.428985 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:49:41.428844 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-serving-cert: secret "maas-api-e2e-shared-a-e321a4-serving-cert" not found Jul 12 22:49:41.428985 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:49:41.428921 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4d24a83-8d62-4cf2-877f-190331f4ed77-maas-api-tls podName:b4d24a83-8d62-4cf2-877f-190331f4ed77 nodeName:}" failed. No retries permitted until 2026-07-12 22:49:41.92890456 +0000 UTC m=+2363.797515366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b4d24a83-8d62-4cf2-877f-190331f4ed77-maas-api-tls") pod "maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" (UID: "b4d24a83-8d62-4cf2-877f-190331f4ed77") : secret "maas-api-e2e-shared-a-e321a4-serving-cert" not found Jul 12 22:49:41.429431 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.429414 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4d24a83-8d62-4cf2-877f-190331f4ed77-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.437979 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.437952 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw2qv\" (UniqueName: \"kubernetes.io/projected/b4d24a83-8d62-4cf2-877f-190331f4ed77-kube-api-access-kw2qv\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.932877 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.932841 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4d24a83-8d62-4cf2-877f-190331f4ed77-maas-api-tls\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:41.935465 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:41.935447 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4d24a83-8d62-4cf2-877f-190331f4ed77-maas-api-tls\") pod \"maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:42.230496 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:42.230409 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:42.370694 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:42.368099 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9"] Jul 12 22:49:42.375270 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:49:42.375238 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d24a83_8d62_4cf2_877f_190331f4ed77.slice/crio-fb589ac6d215276c37a75c2dcad75567602abc6750d055965b405872549763df WatchSource:0}: Error finding container fb589ac6d215276c37a75c2dcad75567602abc6750d055965b405872549763df: Status 404 returned error can't find the container with id fb589ac6d215276c37a75c2dcad75567602abc6750d055965b405872549763df Jul 12 22:49:43.283522 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:43.283482 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" event={"ID":"b4d24a83-8d62-4cf2-877f-190331f4ed77","Type":"ContainerStarted","Data":"40d4c693e7a938822fd6cc65ea3a7c78b04dc38ef8fb0e1efc727585b3cac233"} Jul 12 22:49:43.283730 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:43.283541 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" event={"ID":"b4d24a83-8d62-4cf2-877f-190331f4ed77","Type":"ContainerStarted","Data":"fb589ac6d215276c37a75c2dcad75567602abc6750d055965b405872549763df"} Jul 12 22:49:43.283730 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:43.283633 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:43.302013 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:43.301961 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" podStartSLOduration=1.8630266610000001 podStartE2EDuration="2.301936054s" totalImagesPullingTime="438.909393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:49:43.301520172 +0000 UTC m=+2365.170130998" watchObservedRunningTime="2026-07-12 22:49:43.301936054 +0000 UTC m=+2365.170546879" Jul 12 22:49:49.294736 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.294708 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:49:49.367381 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.367335 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf"] Jul 12 22:49:49.370935 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.370903 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.373706 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.373686 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ecdd13-serving-cert\"" Jul 12 22:49:49.381935 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.381700 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf"] Jul 12 22:49:49.399559 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.399532 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ce77b62-e516-4aa5-b7df-f8163c348093-maas-api-tls\") pod \"maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.399692 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.399567 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ce77b62-e516-4aa5-b7df-f8163c348093-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.399692 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.399598 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vg2q\" (UniqueName: \"kubernetes.io/projected/3ce77b62-e516-4aa5-b7df-f8163c348093-kube-api-access-2vg2q\") pod \"maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.500677 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.500640 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ce77b62-e516-4aa5-b7df-f8163c348093-maas-api-tls\") pod \"maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.500677 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.500680 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ce77b62-e516-4aa5-b7df-f8163c348093-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.500908 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.500705 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vg2q\" (UniqueName: \"kubernetes.io/projected/3ce77b62-e516-4aa5-b7df-f8163c348093-kube-api-access-2vg2q\") pod \"maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.501291 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.501271 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ce77b62-e516-4aa5-b7df-f8163c348093-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.503186 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.503163 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ce77b62-e516-4aa5-b7df-f8163c348093-maas-api-tls\") pod \"maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.508669 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.508645 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vg2q\" (UniqueName: \"kubernetes.io/projected/3ce77b62-e516-4aa5-b7df-f8163c348093-kube-api-access-2vg2q\") pod \"maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.684219 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.684186 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:49.822008 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:49.821976 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf"] Jul 12 22:49:50.319186 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:50.319158 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" event={"ID":"3ce77b62-e516-4aa5-b7df-f8163c348093","Type":"ContainerStarted","Data":"3dcd2e90ace3901d7078486b6d92ba40ae3cf35da34e886927702901f514c678"} Jul 12 22:49:50.319619 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:50.319206 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" event={"ID":"3ce77b62-e516-4aa5-b7df-f8163c348093","Type":"ContainerStarted","Data":"134d4fdda3990f38369640cdff895d46cab3a1e3f6ea220f4a5f62acf98dc674"} Jul 12 22:49:50.319619 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:50.319262 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:49:50.336978 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:50.336922 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" podStartSLOduration=1.040677257 podStartE2EDuration="1.336908611s" totalImagesPullingTime="296.231354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:49:50.334819615 +0000 UTC m=+2372.203430442" watchObservedRunningTime="2026-07-12 22:49:50.336908611 +0000 UTC m=+2372.205519435" Jul 12 22:49:56.329523 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:49:56.329498 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:50:18.800202 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:50:18.800105 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:50:18.813173 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:50:18.806103 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:50:18.813173 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:50:18.806638 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:50:18.813173 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:50:18.813103 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:55:18.835980 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:55:18.835870 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:55:18.842428 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:55:18.842405 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:55:18.842858 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:55:18.842838 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:55:18.849035 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:55:18.849017 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:56:23.011111 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.011072 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7565-140182805183152"] Jul 12 22:56:23.014405 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.014388 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805183152" Jul 12 22:56:23.017207 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.017182 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-9qk5f\"" Jul 12 22:56:23.019760 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.019736 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805183152"] Jul 12 22:56:23.124869 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.124831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gwgb\" (UniqueName: \"kubernetes.io/projected/861c32a7-b4c6-4fb0-a108-961c574dd3b9-kube-api-access-6gwgb\") pod \"test-curl-7565-140182805183152\" (UID: \"861c32a7-b4c6-4fb0-a108-961c574dd3b9\") " pod="openshift-ingress/test-curl-7565-140182805183152" Jul 12 22:56:23.225594 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.225556 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gwgb\" (UniqueName: \"kubernetes.io/projected/861c32a7-b4c6-4fb0-a108-961c574dd3b9-kube-api-access-6gwgb\") pod \"test-curl-7565-140182805183152\" (UID: \"861c32a7-b4c6-4fb0-a108-961c574dd3b9\") " pod="openshift-ingress/test-curl-7565-140182805183152" Jul 12 22:56:23.234395 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.234335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gwgb\" (UniqueName: \"kubernetes.io/projected/861c32a7-b4c6-4fb0-a108-961c574dd3b9-kube-api-access-6gwgb\") pod \"test-curl-7565-140182805183152\" (UID: \"861c32a7-b4c6-4fb0-a108-961c574dd3b9\") " pod="openshift-ingress/test-curl-7565-140182805183152" Jul 12 22:56:23.324776 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.324689 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805183152" Jul 12 22:56:23.459754 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.459730 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805183152"] Jul 12 22:56:23.461499 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:56:23.461469 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod861c32a7_b4c6_4fb0_a108_961c574dd3b9.slice/crio-476690769e9e57f994049256bba0423298578b6068bc9d9af99f236fe961afba WatchSource:0}: Error finding container 476690769e9e57f994049256bba0423298578b6068bc9d9af99f236fe961afba: Status 404 returned error can't find the container with id 476690769e9e57f994049256bba0423298578b6068bc9d9af99f236fe961afba Jul 12 22:56:23.462838 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:23.462820 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:56:24.297047 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:24.297012 2563 generic.go:356] "Generic (PLEG): container finished" podID="861c32a7-b4c6-4fb0-a108-961c574dd3b9" containerID="4da74a032070ff31e82f362cd0dca3e8ae4cbee73ad6b51df4e387a172676d83" exitCode=0 Jul 12 22:56:24.297469 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:24.297053 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805183152" event={"ID":"861c32a7-b4c6-4fb0-a108-961c574dd3b9","Type":"ContainerDied","Data":"4da74a032070ff31e82f362cd0dca3e8ae4cbee73ad6b51df4e387a172676d83"} Jul 12 22:56:24.297469 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:24.297098 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805183152" event={"ID":"861c32a7-b4c6-4fb0-a108-961c574dd3b9","Type":"ContainerStarted","Data":"476690769e9e57f994049256bba0423298578b6068bc9d9af99f236fe961afba"} Jul 12 22:56:25.448058 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.448033 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805183152" Jul 12 22:56:25.482164 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.482136 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7565-140182805183152_861c32a7-b4c6-4fb0-a108-961c574dd3b9/test-curl-7565-140182805183152/0.log" Jul 12 22:56:25.502091 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.502059 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7565-140182805183152"] Jul 12 22:56:25.505431 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.505402 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7565-140182805183152"] Jul 12 22:56:25.542675 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.542640 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gwgb\" (UniqueName: \"kubernetes.io/projected/861c32a7-b4c6-4fb0-a108-961c574dd3b9-kube-api-access-6gwgb\") pod \"861c32a7-b4c6-4fb0-a108-961c574dd3b9\" (UID: \"861c32a7-b4c6-4fb0-a108-961c574dd3b9\") " Jul 12 22:56:25.545053 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.545027 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/861c32a7-b4c6-4fb0-a108-961c574dd3b9-kube-api-access-6gwgb" (OuterVolumeSpecName: "kube-api-access-6gwgb") pod "861c32a7-b4c6-4fb0-a108-961c574dd3b9" (UID: "861c32a7-b4c6-4fb0-a108-961c574dd3b9"). InnerVolumeSpecName "kube-api-access-6gwgb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:56:25.624699 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.624665 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7565-140182805183408"] Jul 12 22:56:25.625030 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.625017 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="861c32a7-b4c6-4fb0-a108-961c574dd3b9" containerName="test-curl-7565-140182805183152" Jul 12 22:56:25.625074 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.625031 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="861c32a7-b4c6-4fb0-a108-961c574dd3b9" containerName="test-curl-7565-140182805183152" Jul 12 22:56:25.625109 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.625097 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="861c32a7-b4c6-4fb0-a108-961c574dd3b9" containerName="test-curl-7565-140182805183152" Jul 12 22:56:25.628213 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.628195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805183408" Jul 12 22:56:25.633423 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.633396 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805183408"] Jul 12 22:56:25.650685 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.644269 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gwgb\" (UniqueName: \"kubernetes.io/projected/861c32a7-b4c6-4fb0-a108-961c574dd3b9-kube-api-access-6gwgb\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:56:25.744793 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.744747 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t76k\" (UniqueName: \"kubernetes.io/projected/5f943ace-4b18-4cae-9c90-0d6b12344938-kube-api-access-5t76k\") pod \"test-curl-7565-140182805183408\" (UID: \"5f943ace-4b18-4cae-9c90-0d6b12344938\") " pod="openshift-ingress/test-curl-7565-140182805183408" Jul 12 22:56:25.845610 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.845566 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t76k\" (UniqueName: \"kubernetes.io/projected/5f943ace-4b18-4cae-9c90-0d6b12344938-kube-api-access-5t76k\") pod \"test-curl-7565-140182805183408\" (UID: \"5f943ace-4b18-4cae-9c90-0d6b12344938\") " pod="openshift-ingress/test-curl-7565-140182805183408" Jul 12 22:56:25.854358 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.854327 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t76k\" (UniqueName: \"kubernetes.io/projected/5f943ace-4b18-4cae-9c90-0d6b12344938-kube-api-access-5t76k\") pod \"test-curl-7565-140182805183408\" (UID: \"5f943ace-4b18-4cae-9c90-0d6b12344938\") " pod="openshift-ingress/test-curl-7565-140182805183408" Jul 12 22:56:25.938419 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:25.938309 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805183408" Jul 12 22:56:26.071934 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:26.071905 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805183408"] Jul 12 22:56:26.072934 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:56:26.072895 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f943ace_4b18_4cae_9c90_0d6b12344938.slice/crio-0643ef9e837ec9dfdec760bf9bd57f3761c07e8c75309c5f9e13785a9d277749 WatchSource:0}: Error finding container 0643ef9e837ec9dfdec760bf9bd57f3761c07e8c75309c5f9e13785a9d277749: Status 404 returned error can't find the container with id 0643ef9e837ec9dfdec760bf9bd57f3761c07e8c75309c5f9e13785a9d277749 Jul 12 22:56:26.312114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:26.312031 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805183152" Jul 12 22:56:26.312114 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:26.312086 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="476690769e9e57f994049256bba0423298578b6068bc9d9af99f236fe961afba" Jul 12 22:56:26.313393 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:26.313347 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805183408" event={"ID":"5f943ace-4b18-4cae-9c90-0d6b12344938","Type":"ContainerStarted","Data":"0643ef9e837ec9dfdec760bf9bd57f3761c07e8c75309c5f9e13785a9d277749"} Jul 12 22:56:26.322944 ip-10-0-134-44 kubenswrapper[2563]: E0712 22:56:26.322914 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7565-140182805183152\" is forbidden: User \"system:node:ip-10-0-134-44.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-134-44.ec2.internal' and this object" podUID="861c32a7-b4c6-4fb0-a108-961c574dd3b9" pod="openshift-ingress/test-curl-7565-140182805183152" Jul 12 22:56:26.631301 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:26.631266 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="861c32a7-b4c6-4fb0-a108-961c574dd3b9" path="/var/lib/kubelet/pods/861c32a7-b4c6-4fb0-a108-961c574dd3b9/volumes" Jul 12 22:56:27.320822 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:27.320787 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f943ace-4b18-4cae-9c90-0d6b12344938" containerID="a5ff7fb5c3c52b046ae7d2fa0be9f586e82cbc3a8762952f7580c2cd2cdaabb2" exitCode=0 Jul 12 22:56:27.321050 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:27.320875 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805183408" event={"ID":"5f943ace-4b18-4cae-9c90-0d6b12344938","Type":"ContainerDied","Data":"a5ff7fb5c3c52b046ae7d2fa0be9f586e82cbc3a8762952f7580c2cd2cdaabb2"} Jul 12 22:56:28.467237 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.467208 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805183408" Jul 12 22:56:28.488067 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.488037 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7565-140182805183408_5f943ace-4b18-4cae-9c90-0d6b12344938/test-curl-7565-140182805183408/0.log" Jul 12 22:56:28.509072 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.509041 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7565-140182805183408"] Jul 12 22:56:28.512433 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.512408 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7565-140182805183408"] Jul 12 22:56:28.564924 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.564890 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t76k\" (UniqueName: \"kubernetes.io/projected/5f943ace-4b18-4cae-9c90-0d6b12344938-kube-api-access-5t76k\") pod \"5f943ace-4b18-4cae-9c90-0d6b12344938\" (UID: \"5f943ace-4b18-4cae-9c90-0d6b12344938\") " Jul 12 22:56:28.567071 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.567037 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f943ace-4b18-4cae-9c90-0d6b12344938-kube-api-access-5t76k" (OuterVolumeSpecName: "kube-api-access-5t76k") pod "5f943ace-4b18-4cae-9c90-0d6b12344938" (UID: "5f943ace-4b18-4cae-9c90-0d6b12344938"). InnerVolumeSpecName "kube-api-access-5t76k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:56:28.630943 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.630913 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f943ace-4b18-4cae-9c90-0d6b12344938" path="/var/lib/kubelet/pods/5f943ace-4b18-4cae-9c90-0d6b12344938/volumes" Jul 12 22:56:28.657568 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.657535 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:28.657905 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.657893 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f943ace-4b18-4cae-9c90-0d6b12344938" containerName="test-curl-7565-140182805183408" Jul 12 22:56:28.657960 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.657906 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f943ace-4b18-4cae-9c90-0d6b12344938" containerName="test-curl-7565-140182805183408" Jul 12 22:56:28.657998 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.657967 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f943ace-4b18-4cae-9c90-0d6b12344938" containerName="test-curl-7565-140182805183408" Jul 12 22:56:28.660900 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.660884 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:28.665662 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.665624 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns8xg\" (UniqueName: \"kubernetes.io/projected/776170e7-b952-4182-ac1d-c058250c3fcf-kube-api-access-ns8xg\") pod \"test-curl-7565-140182805323824\" (UID: \"776170e7-b952-4182-ac1d-c058250c3fcf\") " pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:28.665809 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.665719 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t76k\" (UniqueName: \"kubernetes.io/projected/5f943ace-4b18-4cae-9c90-0d6b12344938-kube-api-access-5t76k\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:56:28.666633 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.666607 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:28.766652 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.766611 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns8xg\" (UniqueName: \"kubernetes.io/projected/776170e7-b952-4182-ac1d-c058250c3fcf-kube-api-access-ns8xg\") pod \"test-curl-7565-140182805323824\" (UID: \"776170e7-b952-4182-ac1d-c058250c3fcf\") " pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:28.775317 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.775290 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns8xg\" (UniqueName: \"kubernetes.io/projected/776170e7-b952-4182-ac1d-c058250c3fcf-kube-api-access-ns8xg\") pod \"test-curl-7565-140182805323824\" (UID: \"776170e7-b952-4182-ac1d-c058250c3fcf\") " pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:28.971259 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:28.971177 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:29.104852 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:29.104830 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:29.107081 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:56:29.107053 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod776170e7_b952_4182_ac1d_c058250c3fcf.slice/crio-04899498e371dd561adde2f7db04e44979ec9a9e2de2a87be7f78f64601515e4 WatchSource:0}: Error finding container 04899498e371dd561adde2f7db04e44979ec9a9e2de2a87be7f78f64601515e4: Status 404 returned error can't find the container with id 04899498e371dd561adde2f7db04e44979ec9a9e2de2a87be7f78f64601515e4 Jul 12 22:56:29.332974 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:29.332884 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805183408" Jul 12 22:56:29.332974 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:29.332893 2563 scope.go:117] "RemoveContainer" containerID="a5ff7fb5c3c52b046ae7d2fa0be9f586e82cbc3a8762952f7580c2cd2cdaabb2" Jul 12 22:56:29.334199 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:29.334076 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805323824" event={"ID":"776170e7-b952-4182-ac1d-c058250c3fcf","Type":"ContainerStarted","Data":"04899498e371dd561adde2f7db04e44979ec9a9e2de2a87be7f78f64601515e4"} Jul 12 22:56:30.342516 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:30.342477 2563 generic.go:356] "Generic (PLEG): container finished" podID="776170e7-b952-4182-ac1d-c058250c3fcf" containerID="9ec624f6c74f8343a61a73441f126eb0867401ece22bf8ee7a3d91eb1768c167" exitCode=0 Jul 12 22:56:30.342971 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:30.342536 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805323824" event={"ID":"776170e7-b952-4182-ac1d-c058250c3fcf","Type":"ContainerDied","Data":"9ec624f6c74f8343a61a73441f126eb0867401ece22bf8ee7a3d91eb1768c167"} Jul 12 22:56:31.490728 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.490703 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:31.509632 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.509608 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7565-140182805323824_776170e7-b952-4182-ac1d-c058250c3fcf/test-curl-7565-140182805323824/0.log" Jul 12 22:56:31.529732 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.529703 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:31.533764 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.533739 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:31.586088 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.586052 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns8xg\" (UniqueName: \"kubernetes.io/projected/776170e7-b952-4182-ac1d-c058250c3fcf-kube-api-access-ns8xg\") pod \"776170e7-b952-4182-ac1d-c058250c3fcf\" (UID: \"776170e7-b952-4182-ac1d-c058250c3fcf\") " Jul 12 22:56:31.588298 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.588271 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/776170e7-b952-4182-ac1d-c058250c3fcf-kube-api-access-ns8xg" (OuterVolumeSpecName: "kube-api-access-ns8xg") pod "776170e7-b952-4182-ac1d-c058250c3fcf" (UID: "776170e7-b952-4182-ac1d-c058250c3fcf"). InnerVolumeSpecName "kube-api-access-ns8xg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:56:31.647655 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.647575 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:31.647928 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.647916 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="776170e7-b952-4182-ac1d-c058250c3fcf" containerName="test-curl-7565-140182805323824" Jul 12 22:56:31.647975 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.647930 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="776170e7-b952-4182-ac1d-c058250c3fcf" containerName="test-curl-7565-140182805323824" Jul 12 22:56:31.648010 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.647992 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="776170e7-b952-4182-ac1d-c058250c3fcf" containerName="test-curl-7565-140182805323824" Jul 12 22:56:31.650865 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.650846 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:31.654868 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.654843 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:31.686583 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.686556 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng5jn\" (UniqueName: \"kubernetes.io/projected/34be0710-eb33-4640-9d0f-0752966f1ac4-kube-api-access-ng5jn\") pod \"test-curl-7565-140182805323824\" (UID: \"34be0710-eb33-4640-9d0f-0752966f1ac4\") " pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:31.686732 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.686618 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns8xg\" (UniqueName: \"kubernetes.io/projected/776170e7-b952-4182-ac1d-c058250c3fcf-kube-api-access-ns8xg\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:56:31.787774 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.787731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng5jn\" (UniqueName: \"kubernetes.io/projected/34be0710-eb33-4640-9d0f-0752966f1ac4-kube-api-access-ng5jn\") pod \"test-curl-7565-140182805323824\" (UID: \"34be0710-eb33-4640-9d0f-0752966f1ac4\") " pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:31.797132 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.797101 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng5jn\" (UniqueName: \"kubernetes.io/projected/34be0710-eb33-4640-9d0f-0752966f1ac4-kube-api-access-ng5jn\") pod \"test-curl-7565-140182805323824\" (UID: \"34be0710-eb33-4640-9d0f-0752966f1ac4\") " pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:31.962245 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:31.962155 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:32.099981 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:32.099956 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:32.101922 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:56:32.101893 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34be0710_eb33_4640_9d0f_0752966f1ac4.slice/crio-da9b43c96fae1f45063d63f8203ad23a33f8f45bf08958ddb94da6812acb8ee6 WatchSource:0}: Error finding container da9b43c96fae1f45063d63f8203ad23a33f8f45bf08958ddb94da6812acb8ee6: Status 404 returned error can't find the container with id da9b43c96fae1f45063d63f8203ad23a33f8f45bf08958ddb94da6812acb8ee6 Jul 12 22:56:32.352867 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:32.352835 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805323824" event={"ID":"34be0710-eb33-4640-9d0f-0752966f1ac4","Type":"ContainerStarted","Data":"da9b43c96fae1f45063d63f8203ad23a33f8f45bf08958ddb94da6812acb8ee6"} Jul 12 22:56:32.354099 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:32.354076 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04899498e371dd561adde2f7db04e44979ec9a9e2de2a87be7f78f64601515e4" Jul 12 22:56:32.354217 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:32.354116 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:32.366023 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:32.365994 2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7565-140182805323824" oldPodUID="776170e7-b952-4182-ac1d-c058250c3fcf" podUID="34be0710-eb33-4640-9d0f-0752966f1ac4" Jul 12 22:56:32.630741 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:32.630658 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="776170e7-b952-4182-ac1d-c058250c3fcf" path="/var/lib/kubelet/pods/776170e7-b952-4182-ac1d-c058250c3fcf/volumes" Jul 12 22:56:33.360780 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:33.360748 2563 generic.go:356] "Generic (PLEG): container finished" podID="34be0710-eb33-4640-9d0f-0752966f1ac4" containerID="a8433846290405452af9ecb3041239215bb5f398a99bbd53526b51f88e2b1c32" exitCode=0 Jul 12 22:56:33.360966 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:33.360791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805323824" event={"ID":"34be0710-eb33-4640-9d0f-0752966f1ac4","Type":"ContainerDied","Data":"a8433846290405452af9ecb3041239215bb5f398a99bbd53526b51f88e2b1c32"} Jul 12 22:56:34.510606 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.510583 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:34.528916 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.528890 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7565-140182805323824_34be0710-eb33-4640-9d0f-0752966f1ac4/test-curl-7565-140182805323824/0.log" Jul 12 22:56:34.550415 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.550383 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:34.552481 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.552458 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7565-140182805323824"] Jul 12 22:56:34.610453 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.607611 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng5jn\" (UniqueName: \"kubernetes.io/projected/34be0710-eb33-4640-9d0f-0752966f1ac4-kube-api-access-ng5jn\") pod \"34be0710-eb33-4640-9d0f-0752966f1ac4\" (UID: \"34be0710-eb33-4640-9d0f-0752966f1ac4\") " Jul 12 22:56:34.613382 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.610993 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34be0710-eb33-4640-9d0f-0752966f1ac4-kube-api-access-ng5jn" (OuterVolumeSpecName: "kube-api-access-ng5jn") pod "34be0710-eb33-4640-9d0f-0752966f1ac4" (UID: "34be0710-eb33-4640-9d0f-0752966f1ac4"). InnerVolumeSpecName "kube-api-access-ng5jn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:56:34.631905 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.631880 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34be0710-eb33-4640-9d0f-0752966f1ac4" path="/var/lib/kubelet/pods/34be0710-eb33-4640-9d0f-0752966f1ac4/volumes" Jul 12 22:56:34.668301 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.668271 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:34.668635 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.668623 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34be0710-eb33-4640-9d0f-0752966f1ac4" containerName="test-curl-7565-140182805323824" Jul 12 22:56:34.668677 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.668636 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34be0710-eb33-4640-9d0f-0752966f1ac4" containerName="test-curl-7565-140182805323824" Jul 12 22:56:34.668725 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.668701 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34be0710-eb33-4640-9d0f-0752966f1ac4" containerName="test-curl-7565-140182805323824" Jul 12 22:56:34.671790 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.671775 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:34.675079 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.675057 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:34.708679 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.708650 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvtdh\" (UniqueName: \"kubernetes.io/projected/06b12729-6b16-487f-a5bb-2ad2a63063e5-kube-api-access-zvtdh\") pod \"test-curl-7565-140182805324832\" (UID: \"06b12729-6b16-487f-a5bb-2ad2a63063e5\") " pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:34.708808 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.708735 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng5jn\" (UniqueName: \"kubernetes.io/projected/34be0710-eb33-4640-9d0f-0752966f1ac4-kube-api-access-ng5jn\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:56:34.809239 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.809207 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvtdh\" (UniqueName: \"kubernetes.io/projected/06b12729-6b16-487f-a5bb-2ad2a63063e5-kube-api-access-zvtdh\") pod \"test-curl-7565-140182805324832\" (UID: \"06b12729-6b16-487f-a5bb-2ad2a63063e5\") " pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:34.816838 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.816809 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvtdh\" (UniqueName: \"kubernetes.io/projected/06b12729-6b16-487f-a5bb-2ad2a63063e5-kube-api-access-zvtdh\") pod \"test-curl-7565-140182805324832\" (UID: \"06b12729-6b16-487f-a5bb-2ad2a63063e5\") " pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:34.982532 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:34.982448 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:35.111728 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:35.111703 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:35.113411 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:56:35.113384 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06b12729_6b16_487f_a5bb_2ad2a63063e5.slice/crio-021535f688e1bd6a0e7b71532c42f73f9469d7998db7a91e46b75f4833217275 WatchSource:0}: Error finding container 021535f688e1bd6a0e7b71532c42f73f9469d7998db7a91e46b75f4833217275: Status 404 returned error can't find the container with id 021535f688e1bd6a0e7b71532c42f73f9469d7998db7a91e46b75f4833217275 Jul 12 22:56:35.371963 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:35.371935 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805324832" event={"ID":"06b12729-6b16-487f-a5bb-2ad2a63063e5","Type":"ContainerStarted","Data":"021535f688e1bd6a0e7b71532c42f73f9469d7998db7a91e46b75f4833217275"} Jul 12 22:56:35.373097 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:35.373075 2563 scope.go:117] "RemoveContainer" containerID="a8433846290405452af9ecb3041239215bb5f398a99bbd53526b51f88e2b1c32" Jul 12 22:56:35.373097 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:35.373090 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805323824" Jul 12 22:56:36.379960 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:36.379921 2563 generic.go:356] "Generic (PLEG): container finished" podID="06b12729-6b16-487f-a5bb-2ad2a63063e5" containerID="368a79ef96100e888897f13d8a46b674d6c4ded87afa59cc5b2979cd4a5677ef" exitCode=0 Jul 12 22:56:36.380338 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:36.380008 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805324832" event={"ID":"06b12729-6b16-487f-a5bb-2ad2a63063e5","Type":"ContainerDied","Data":"368a79ef96100e888897f13d8a46b674d6c4ded87afa59cc5b2979cd4a5677ef"} Jul 12 22:56:37.543894 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.543873 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:37.562006 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.561984 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7565-140182805324832_06b12729-6b16-487f-a5bb-2ad2a63063e5/test-curl-7565-140182805324832/0.log" Jul 12 22:56:37.583810 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.583785 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:37.587910 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.587886 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:37.630052 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.630025 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvtdh\" (UniqueName: \"kubernetes.io/projected/06b12729-6b16-487f-a5bb-2ad2a63063e5-kube-api-access-zvtdh\") pod \"06b12729-6b16-487f-a5bb-2ad2a63063e5\" (UID: \"06b12729-6b16-487f-a5bb-2ad2a63063e5\") " Jul 12 22:56:37.632189 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.632164 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b12729-6b16-487f-a5bb-2ad2a63063e5-kube-api-access-zvtdh" (OuterVolumeSpecName: "kube-api-access-zvtdh") pod "06b12729-6b16-487f-a5bb-2ad2a63063e5" (UID: "06b12729-6b16-487f-a5bb-2ad2a63063e5"). InnerVolumeSpecName "kube-api-access-zvtdh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:56:37.704015 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.703985 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:37.704325 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.704314 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b12729-6b16-487f-a5bb-2ad2a63063e5" containerName="test-curl-7565-140182805324832" Jul 12 22:56:37.704393 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.704327 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b12729-6b16-487f-a5bb-2ad2a63063e5" containerName="test-curl-7565-140182805324832" Jul 12 22:56:37.704437 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.704405 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b12729-6b16-487f-a5bb-2ad2a63063e5" containerName="test-curl-7565-140182805324832" Jul 12 22:56:37.707315 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.707298 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:37.712903 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.712876 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:37.730858 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.730834 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwdmf\" (UniqueName: \"kubernetes.io/projected/8823614d-e603-47ad-ab82-96cb032d4301-kube-api-access-gwdmf\") pod \"test-curl-7565-140182805324832\" (UID: \"8823614d-e603-47ad-ab82-96cb032d4301\") " pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:37.731000 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.730984 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvtdh\" (UniqueName: \"kubernetes.io/projected/06b12729-6b16-487f-a5bb-2ad2a63063e5-kube-api-access-zvtdh\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:56:37.832481 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.832449 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwdmf\" (UniqueName: \"kubernetes.io/projected/8823614d-e603-47ad-ab82-96cb032d4301-kube-api-access-gwdmf\") pod \"test-curl-7565-140182805324832\" (UID: \"8823614d-e603-47ad-ab82-96cb032d4301\") " pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:37.839751 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:37.839722 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwdmf\" (UniqueName: \"kubernetes.io/projected/8823614d-e603-47ad-ab82-96cb032d4301-kube-api-access-gwdmf\") pod \"test-curl-7565-140182805324832\" (UID: \"8823614d-e603-47ad-ab82-96cb032d4301\") " pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:38.017284 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:38.017202 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:38.151147 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:38.151117 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:38.153092 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:56:38.153062 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8823614d_e603_47ad_ab82_96cb032d4301.slice/crio-5ec6f8b31cb027741da550703e4a302589535579747be0e434506d7d09f171ad WatchSource:0}: Error finding container 5ec6f8b31cb027741da550703e4a302589535579747be0e434506d7d09f171ad: Status 404 returned error can't find the container with id 5ec6f8b31cb027741da550703e4a302589535579747be0e434506d7d09f171ad Jul 12 22:56:38.391862 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:38.391836 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:38.392050 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:38.391839 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="021535f688e1bd6a0e7b71532c42f73f9469d7998db7a91e46b75f4833217275" Jul 12 22:56:38.399239 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:38.399206 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805324832" event={"ID":"8823614d-e603-47ad-ab82-96cb032d4301","Type":"ContainerStarted","Data":"5ec6f8b31cb027741da550703e4a302589535579747be0e434506d7d09f171ad"} Jul 12 22:56:38.409272 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:38.409245 2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7565-140182805324832" oldPodUID="06b12729-6b16-487f-a5bb-2ad2a63063e5" podUID="8823614d-e603-47ad-ab82-96cb032d4301" Jul 12 22:56:38.632734 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:38.632703 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06b12729-6b16-487f-a5bb-2ad2a63063e5" path="/var/lib/kubelet/pods/06b12729-6b16-487f-a5bb-2ad2a63063e5/volumes" Jul 12 22:56:39.405569 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:39.405536 2563 generic.go:356] "Generic (PLEG): container finished" podID="8823614d-e603-47ad-ab82-96cb032d4301" containerID="e031a118a5b058271cd1117823a7f2b4ba92c73fe6402cb13d425037cc1614de" exitCode=0 Jul 12 22:56:39.405715 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:39.405600 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7565-140182805324832" event={"ID":"8823614d-e603-47ad-ab82-96cb032d4301","Type":"ContainerDied","Data":"e031a118a5b058271cd1117823a7f2b4ba92c73fe6402cb13d425037cc1614de"} Jul 12 22:56:40.557295 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:40.557270 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:40.575059 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:40.575035 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7565-140182805324832_8823614d-e603-47ad-ab82-96cb032d4301/test-curl-7565-140182805324832/0.log" Jul 12 22:56:40.598153 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:40.598123 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:40.600522 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:40.600500 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7565-140182805324832"] Jul 12 22:56:40.656511 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:40.656490 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwdmf\" (UniqueName: \"kubernetes.io/projected/8823614d-e603-47ad-ab82-96cb032d4301-kube-api-access-gwdmf\") pod \"8823614d-e603-47ad-ab82-96cb032d4301\" (UID: \"8823614d-e603-47ad-ab82-96cb032d4301\") " Jul 12 22:56:40.658575 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:40.658547 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8823614d-e603-47ad-ab82-96cb032d4301-kube-api-access-gwdmf" (OuterVolumeSpecName: "kube-api-access-gwdmf") pod "8823614d-e603-47ad-ab82-96cb032d4301" (UID: "8823614d-e603-47ad-ab82-96cb032d4301"). InnerVolumeSpecName "kube-api-access-gwdmf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:56:40.757336 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:40.757305 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwdmf\" (UniqueName: \"kubernetes.io/projected/8823614d-e603-47ad-ab82-96cb032d4301-kube-api-access-gwdmf\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:56:41.416852 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:41.416825 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7565-140182805324832" Jul 12 22:56:41.417028 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:41.416844 2563 scope.go:117] "RemoveContainer" containerID="e031a118a5b058271cd1117823a7f2b4ba92c73fe6402cb13d425037cc1614de" Jul 12 22:56:42.631179 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:42.631146 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8823614d-e603-47ad-ab82-96cb032d4301" path="/var/lib/kubelet/pods/8823614d-e603-47ad-ab82-96cb032d4301/volumes" Jul 12 22:56:51.221389 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.221322 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9"] Jul 12 22:56:51.221981 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.221818 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" podUID="b4d24a83-8d62-4cf2-877f-190331f4ed77" containerName="maas-api" containerID="cri-o://40d4c693e7a938822fd6cc65ea3a7c78b04dc38ef8fb0e1efc727585b3cac233" gracePeriod=30 Jul 12 22:56:51.473142 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.473072 2563 generic.go:356] "Generic (PLEG): container finished" podID="b4d24a83-8d62-4cf2-877f-190331f4ed77" containerID="40d4c693e7a938822fd6cc65ea3a7c78b04dc38ef8fb0e1efc727585b3cac233" exitCode=0 Jul 12 22:56:51.473260 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.473152 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" event={"ID":"b4d24a83-8d62-4cf2-877f-190331f4ed77","Type":"ContainerDied","Data":"40d4c693e7a938822fd6cc65ea3a7c78b04dc38ef8fb0e1efc727585b3cac233"} Jul 12 22:56:51.473260 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.473208 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" event={"ID":"b4d24a83-8d62-4cf2-877f-190331f4ed77","Type":"ContainerDied","Data":"fb589ac6d215276c37a75c2dcad75567602abc6750d055965b405872549763df"} Jul 12 22:56:51.473260 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.473224 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb589ac6d215276c37a75c2dcad75567602abc6750d055965b405872549763df" Jul 12 22:56:51.477315 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.477297 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:56:51.550981 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.550957 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4d24a83-8d62-4cf2-877f-190331f4ed77-maas-api-tls\") pod \"b4d24a83-8d62-4cf2-877f-190331f4ed77\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " Jul 12 22:56:51.551129 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.550989 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw2qv\" (UniqueName: \"kubernetes.io/projected/b4d24a83-8d62-4cf2-877f-190331f4ed77-kube-api-access-kw2qv\") pod \"b4d24a83-8d62-4cf2-877f-190331f4ed77\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " Jul 12 22:56:51.551129 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.551119 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4d24a83-8d62-4cf2-877f-190331f4ed77-openshift-service-ca\") pod \"b4d24a83-8d62-4cf2-877f-190331f4ed77\" (UID: \"b4d24a83-8d62-4cf2-877f-190331f4ed77\") " Jul 12 22:56:51.551509 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.551477 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4d24a83-8d62-4cf2-877f-190331f4ed77-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4d24a83-8d62-4cf2-877f-190331f4ed77" (UID: "b4d24a83-8d62-4cf2-877f-190331f4ed77"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:56:51.553133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.553113 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4d24a83-8d62-4cf2-877f-190331f4ed77-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4d24a83-8d62-4cf2-877f-190331f4ed77" (UID: "b4d24a83-8d62-4cf2-877f-190331f4ed77"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:56:51.553133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.553127 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4d24a83-8d62-4cf2-877f-190331f4ed77-kube-api-access-kw2qv" (OuterVolumeSpecName: "kube-api-access-kw2qv") pod "b4d24a83-8d62-4cf2-877f-190331f4ed77" (UID: "b4d24a83-8d62-4cf2-877f-190331f4ed77"). InnerVolumeSpecName "kube-api-access-kw2qv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:56:51.652755 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.652728 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4d24a83-8d62-4cf2-877f-190331f4ed77-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:56:51.652755 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.652751 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4d24a83-8d62-4cf2-877f-190331f4ed77-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:56:51.652755 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:51.652761 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw2qv\" (UniqueName: \"kubernetes.io/projected/b4d24a83-8d62-4cf2-877f-190331f4ed77-kube-api-access-kw2qv\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:56:52.478523 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:52.478495 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9" Jul 12 22:56:52.497694 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:52.497668 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9"] Jul 12 22:56:52.501669 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:52.501644 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e321a4-5666db7f6b-wdfl9"] Jul 12 22:56:52.630923 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:56:52.630881 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4d24a83-8d62-4cf2-877f-190331f4ed77" path="/var/lib/kubelet/pods/b4d24a83-8d62-4cf2-877f-190331f4ed77/volumes" Jul 12 22:57:17.369931 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.369891 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf"] Jul 12 22:57:17.370529 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.370137 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" podUID="3ce77b62-e516-4aa5-b7df-f8163c348093" containerName="maas-api" containerID="cri-o://3dcd2e90ace3901d7078486b6d92ba40ae3cf35da34e886927702901f514c678" gracePeriod=30 Jul 12 22:57:17.608782 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.608754 2563 generic.go:356] "Generic (PLEG): container finished" podID="3ce77b62-e516-4aa5-b7df-f8163c348093" containerID="3dcd2e90ace3901d7078486b6d92ba40ae3cf35da34e886927702901f514c678" exitCode=0 Jul 12 22:57:17.608924 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.608826 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" event={"ID":"3ce77b62-e516-4aa5-b7df-f8163c348093","Type":"ContainerDied","Data":"3dcd2e90ace3901d7078486b6d92ba40ae3cf35da34e886927702901f514c678"} Jul 12 22:57:17.628225 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.628164 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:57:17.788993 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.788958 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ce77b62-e516-4aa5-b7df-f8163c348093-openshift-service-ca\") pod \"3ce77b62-e516-4aa5-b7df-f8163c348093\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " Jul 12 22:57:17.789168 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.789011 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ce77b62-e516-4aa5-b7df-f8163c348093-maas-api-tls\") pod \"3ce77b62-e516-4aa5-b7df-f8163c348093\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " Jul 12 22:57:17.789168 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.789051 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vg2q\" (UniqueName: \"kubernetes.io/projected/3ce77b62-e516-4aa5-b7df-f8163c348093-kube-api-access-2vg2q\") pod \"3ce77b62-e516-4aa5-b7df-f8163c348093\" (UID: \"3ce77b62-e516-4aa5-b7df-f8163c348093\") " Jul 12 22:57:17.789345 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.789323 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ce77b62-e516-4aa5-b7df-f8163c348093-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3ce77b62-e516-4aa5-b7df-f8163c348093" (UID: "3ce77b62-e516-4aa5-b7df-f8163c348093"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:57:17.791466 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.791429 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ce77b62-e516-4aa5-b7df-f8163c348093-kube-api-access-2vg2q" (OuterVolumeSpecName: "kube-api-access-2vg2q") pod "3ce77b62-e516-4aa5-b7df-f8163c348093" (UID: "3ce77b62-e516-4aa5-b7df-f8163c348093"). InnerVolumeSpecName "kube-api-access-2vg2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:57:17.791585 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.791468 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ce77b62-e516-4aa5-b7df-f8163c348093-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3ce77b62-e516-4aa5-b7df-f8163c348093" (UID: "3ce77b62-e516-4aa5-b7df-f8163c348093"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:57:17.890007 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.889924 2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ce77b62-e516-4aa5-b7df-f8163c348093-openshift-service-ca\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:57:17.890007 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.889955 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ce77b62-e516-4aa5-b7df-f8163c348093-maas-api-tls\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:57:17.890007 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:17.889964 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vg2q\" (UniqueName: \"kubernetes.io/projected/3ce77b62-e516-4aa5-b7df-f8163c348093-kube-api-access-2vg2q\") on node \"ip-10-0-134-44.ec2.internal\" DevicePath \"\"" Jul 12 22:57:18.616309 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:18.616279 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" Jul 12 22:57:18.616760 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:18.616305 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf" event={"ID":"3ce77b62-e516-4aa5-b7df-f8163c348093","Type":"ContainerDied","Data":"134d4fdda3990f38369640cdff895d46cab3a1e3f6ea220f4a5f62acf98dc674"} Jul 12 22:57:18.616760 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:18.616406 2563 scope.go:117] "RemoveContainer" containerID="3dcd2e90ace3901d7078486b6d92ba40ae3cf35da34e886927702901f514c678" Jul 12 22:57:18.636703 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:18.636680 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf"] Jul 12 22:57:18.640305 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:18.640282 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ecdd13-5f65bfb557-bl6jf"] Jul 12 22:57:18.951801 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:18.951717 2563 scope.go:117] "RemoveContainer" containerID="40d4c693e7a938822fd6cc65ea3a7c78b04dc38ef8fb0e1efc727585b3cac233" Jul 12 22:57:20.630925 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:20.630892 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ce77b62-e516-4aa5-b7df-f8163c348093" path="/var/lib/kubelet/pods/3ce77b62-e516-4aa5-b7df-f8163c348093/volumes" Jul 12 22:57:38.243707 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:38.243676 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d8865b84c-gjw92_fb8903d6-da56-485b-bbbe-98ebc583b80f/manager/0.log" Jul 12 22:57:38.580640 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:38.580567 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56d7986bbd-47jnj_33065002-dab1-48fc-9b08-a4eee34cd6b3/copy-manifests/0.log" Jul 12 22:57:38.586862 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:38.586844 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56d7986bbd-47jnj_33065002-dab1-48fc-9b08-a4eee34cd6b3/manager/0.log" Jul 12 22:57:39.181378 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:39.181336 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-74ffd49cf8-ln7tn_6860ac2c-c2bb-45b3-b21c-464ca22f35cf/maas-api/0.log" Jul 12 22:57:40.812811 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:40.812783 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8v6pd_9a510e49-449c-4e63-a9a5-94fae278cd96/postgres/0.log" Jul 12 22:57:41.600834 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.600805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749_3e305a11-031b-4458-a56c-ee28f33337cf/util/0.log" Jul 12 22:57:41.606525 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.606493 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749_3e305a11-031b-4458-a56c-ee28f33337cf/pull/0.log" Jul 12 22:57:41.611684 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.611667 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749_3e305a11-031b-4458-a56c-ee28f33337cf/extract/0.log" Jul 12 22:57:41.725540 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.725512 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb_d2020f75-296f-42a0-b222-942007ead610/util/0.log" Jul 12 22:57:41.731311 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.731289 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb_d2020f75-296f-42a0-b222-942007ead610/pull/0.log" Jul 12 22:57:41.736297 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.736274 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb_d2020f75-296f-42a0-b222-942007ead610/extract/0.log" Jul 12 22:57:41.852602 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.852510 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn_b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9/extract/0.log" Jul 12 22:57:41.858235 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.858207 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn_b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9/util/0.log" Jul 12 22:57:41.863613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.863595 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn_b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9/pull/0.log" Jul 12 22:57:41.982035 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.982010 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg_a8fe40c3-1c50-48bb-b383-ceecc64a0ad7/extract/0.log" Jul 12 22:57:41.988142 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.988112 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg_a8fe40c3-1c50-48bb-b383-ceecc64a0ad7/util/0.log" Jul 12 22:57:41.995529 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:41.995506 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg_a8fe40c3-1c50-48bb-b383-ceecc64a0ad7/pull/0.log" Jul 12 22:57:43.518030 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:43.518000 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747db99d8-78dk4_938ac42e-556c-483d-9306-cd0944e21be2/kube-auth-proxy/0.log" Jul 12 22:57:43.975103 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:43.975077 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-dsjgn_c82806ac-a7c2-4f98-9663-d8cacfd2e32a/payload-processing/0.log" Jul 12 22:57:44.443394 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:44.443345 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-ndjjx_ed3b6cdf-5dff-4305-a75b-aa447323ccff/main/0.log" Jul 12 22:57:44.663133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:44.663096 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-gd22g_64d626a9-fcfb-42f4-8422-33bc8dc75cb2/main/0.log" Jul 12 22:57:45.001823 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:45.001794 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-f6vlt_972b4f9b-23be-491c-aeaf-92190e88c90e/main/0.log" Jul 12 22:57:49.404717 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.404682 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xkcz8/must-gather-p9qd2"] Jul 12 22:57:49.405194 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.405136 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce77b62-e516-4aa5-b7df-f8163c348093" containerName="maas-api" Jul 12 22:57:49.405194 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.405153 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce77b62-e516-4aa5-b7df-f8163c348093" containerName="maas-api" Jul 12 22:57:49.405194 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.405172 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d24a83-8d62-4cf2-877f-190331f4ed77" containerName="maas-api" Jul 12 22:57:49.405194 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.405179 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d24a83-8d62-4cf2-877f-190331f4ed77" containerName="maas-api" Jul 12 22:57:49.405461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.405210 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8823614d-e603-47ad-ab82-96cb032d4301" containerName="test-curl-7565-140182805324832" Jul 12 22:57:49.405461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.405220 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8823614d-e603-47ad-ab82-96cb032d4301" containerName="test-curl-7565-140182805324832" Jul 12 22:57:49.405461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.405311 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4d24a83-8d62-4cf2-877f-190331f4ed77" containerName="maas-api" Jul 12 22:57:49.405461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.405323 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ce77b62-e516-4aa5-b7df-f8163c348093" containerName="maas-api" Jul 12 22:57:49.405461 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.405335 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8823614d-e603-47ad-ab82-96cb032d4301" containerName="test-curl-7565-140182805324832" Jul 12 22:57:49.408924 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.408900 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkcz8/must-gather-p9qd2" Jul 12 22:57:49.411495 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.411469 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkcz8\"/\"openshift-service-ca.crt\"" Jul 12 22:57:49.412253 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.412231 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xkcz8\"/\"default-dockercfg-kqqvs\"" Jul 12 22:57:49.412253 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.412245 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xkcz8\"/\"kube-root-ca.crt\"" Jul 12 22:57:49.420779 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.418099 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkcz8/must-gather-p9qd2"] Jul 12 22:57:49.446410 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.446339 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q747\" (UniqueName: \"kubernetes.io/projected/b33d5966-6323-46da-8842-799629108288-kube-api-access-7q747\") pod \"must-gather-p9qd2\" (UID: \"b33d5966-6323-46da-8842-799629108288\") " pod="openshift-must-gather-xkcz8/must-gather-p9qd2" Jul 12 22:57:49.446593 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.446415 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b33d5966-6323-46da-8842-799629108288-must-gather-output\") pod \"must-gather-p9qd2\" (UID: \"b33d5966-6323-46da-8842-799629108288\") " pod="openshift-must-gather-xkcz8/must-gather-p9qd2" Jul 12 22:57:49.547466 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.547418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q747\" (UniqueName: \"kubernetes.io/projected/b33d5966-6323-46da-8842-799629108288-kube-api-access-7q747\") pod \"must-gather-p9qd2\" (UID: \"b33d5966-6323-46da-8842-799629108288\") " pod="openshift-must-gather-xkcz8/must-gather-p9qd2" Jul 12 22:57:49.547466 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.547473 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b33d5966-6323-46da-8842-799629108288-must-gather-output\") pod \"must-gather-p9qd2\" (UID: \"b33d5966-6323-46da-8842-799629108288\") " pod="openshift-must-gather-xkcz8/must-gather-p9qd2" Jul 12 22:57:49.547795 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.547779 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b33d5966-6323-46da-8842-799629108288-must-gather-output\") pod \"must-gather-p9qd2\" (UID: \"b33d5966-6323-46da-8842-799629108288\") " pod="openshift-must-gather-xkcz8/must-gather-p9qd2" Jul 12 22:57:49.555532 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.555512 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q747\" (UniqueName: \"kubernetes.io/projected/b33d5966-6323-46da-8842-799629108288-kube-api-access-7q747\") pod \"must-gather-p9qd2\" (UID: \"b33d5966-6323-46da-8842-799629108288\") " pod="openshift-must-gather-xkcz8/must-gather-p9qd2" Jul 12 22:57:49.721316 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.721220 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkcz8/must-gather-p9qd2" Jul 12 22:57:49.862298 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:49.862252 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkcz8/must-gather-p9qd2"] Jul 12 22:57:49.864764 ip-10-0-134-44 kubenswrapper[2563]: W0712 22:57:49.864733 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb33d5966_6323_46da_8842_799629108288.slice/crio-990ad5501e45a6bbb32ebcd7a0ed69513f96c3a914af85927f47f503ca102eae WatchSource:0}: Error finding container 990ad5501e45a6bbb32ebcd7a0ed69513f96c3a914af85927f47f503ca102eae: Status 404 returned error can't find the container with id 990ad5501e45a6bbb32ebcd7a0ed69513f96c3a914af85927f47f503ca102eae Jul 12 22:57:50.788528 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:50.788446 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkcz8/must-gather-p9qd2" event={"ID":"b33d5966-6323-46da-8842-799629108288","Type":"ContainerStarted","Data":"85063edb9966476418bbbed4a4df5d61ab1bf8793cbc4e208d9bbc256bd7402f"} Jul 12 22:57:50.788528 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:50.788498 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkcz8/must-gather-p9qd2" event={"ID":"b33d5966-6323-46da-8842-799629108288","Type":"ContainerStarted","Data":"990ad5501e45a6bbb32ebcd7a0ed69513f96c3a914af85927f47f503ca102eae"} Jul 12 22:57:51.796746 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:51.796709 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkcz8/must-gather-p9qd2" event={"ID":"b33d5966-6323-46da-8842-799629108288","Type":"ContainerStarted","Data":"2c4434e2f911c222f03be4ba86c1ccb23d4dee8acd573a05550ee5e00350e8f5"} Jul 12 22:57:51.811541 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:51.811473 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xkcz8/must-gather-p9qd2" podStartSLOduration=2.045269885 podStartE2EDuration="2.811444136s" totalImagesPullingTime="766.174251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:57:51.810200565 +0000 UTC m=+2853.678811390" watchObservedRunningTime="2026-07-12 22:57:51.811444136 +0000 UTC m=+2853.680054961" Jul 12 22:57:52.214686 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:52.214639 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cd9nz_87116643-5000-4846-a88b-a452ab0ef89e/global-pull-secret-syncer/0.log" Jul 12 22:57:52.338916 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:52.338888 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g7vtm_1cbd4733-347d-40b6-a124-7f8df70f0ec6/konnectivity-agent/0.log" Jul 12 22:57:52.390304 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:52.390267 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-44.ec2.internal_73621ab645bf612f0f90ffad1d6de781/haproxy/0.log" Jul 12 22:57:56.678514 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.678478 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749_3e305a11-031b-4458-a56c-ee28f33337cf/extract/0.log" Jul 12 22:57:56.705533 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.705500 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749_3e305a11-031b-4458-a56c-ee28f33337cf/util/0.log" Jul 12 22:57:56.727004 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.726972 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tf749_3e305a11-031b-4458-a56c-ee28f33337cf/pull/0.log" Jul 12 22:57:56.762261 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.762230 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb_d2020f75-296f-42a0-b222-942007ead610/extract/0.log" Jul 12 22:57:56.785870 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.785836 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb_d2020f75-296f-42a0-b222-942007ead610/util/0.log" Jul 12 22:57:56.811761 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.811720 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ccdwb_d2020f75-296f-42a0-b222-942007ead610/pull/0.log" Jul 12 22:57:56.846522 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.846493 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn_b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9/extract/0.log" Jul 12 22:57:56.870060 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.870029 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn_b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9/util/0.log" Jul 12 22:57:56.894538 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.894511 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fx9rn_b20e11d5-636c-4fe6-a3a8-6a0f0eb610b9/pull/0.log" Jul 12 22:57:56.925669 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.925643 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg_a8fe40c3-1c50-48bb-b383-ceecc64a0ad7/extract/0.log" Jul 12 22:57:56.949282 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.949210 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg_a8fe40c3-1c50-48bb-b383-ceecc64a0ad7/util/0.log" Jul 12 22:57:56.975080 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:56.975020 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pl6kg_a8fe40c3-1c50-48bb-b383-ceecc64a0ad7/pull/0.log" Jul 12 22:57:59.608833 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:59.608799 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mgfgq_67fb80af-f51e-4c1e-8c22-a6b2869e0f9e/node-exporter/0.log" Jul 12 22:57:59.631337 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:59.631282 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mgfgq_67fb80af-f51e-4c1e-8c22-a6b2869e0f9e/kube-rbac-proxy/0.log" Jul 12 22:57:59.654156 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:57:59.654130 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mgfgq_67fb80af-f51e-4c1e-8c22-a6b2869e0f9e/init-textfile/0.log" Jul 12 22:58:02.226735 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:02.226674 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/1.log" Jul 12 22:58:02.234676 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:02.234638 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jx469_1dcff9c3-da5f-47d9-a082-7a79e9adad93/console-operator/2.log" Jul 12 22:58:02.797714 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:02.797678 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-clt97_b6a3d700-67f4-4928-be3a-89f5419a8eb5/download-server/0.log" Jul 12 22:58:02.976458 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:02.976420 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r"] Jul 12 22:58:02.982133 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:02.982108 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:02.987311 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:02.987282 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r"] Jul 12 22:58:03.088687 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.088643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-sys\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.088875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.088703 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-podres\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.088875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.088752 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-lib-modules\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.088875 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.088791 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-proc\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.089042 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.088874 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z28dw\" (UniqueName: \"kubernetes.io/projected/e733eb5d-1245-42fc-b269-e67f1c712206-kube-api-access-z28dw\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.189999 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.189961 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z28dw\" (UniqueName: \"kubernetes.io/projected/e733eb5d-1245-42fc-b269-e67f1c712206-kube-api-access-z28dw\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.190195 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.190036 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-sys\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.190195 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.190062 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-podres\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.190195 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.190094 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-lib-modules\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.190195 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.190127 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-proc\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.190429 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.190202 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-proc\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.190429 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.190199 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-sys\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.190429 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.190257 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-podres\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.190429 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.190303 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e733eb5d-1245-42fc-b269-e67f1c712206-lib-modules\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.198817 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.198788 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z28dw\" (UniqueName: \"kubernetes.io/projected/e733eb5d-1245-42fc-b269-e67f1c712206-kube-api-access-z28dw\") pod \"perf-node-gather-daemonset-hdd4r\" (UID: \"e733eb5d-1245-42fc-b269-e67f1c712206\") " pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.295512 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.295471 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.476210 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.476175 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r"] Jul 12 22:58:03.894614 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.894579 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" event={"ID":"e733eb5d-1245-42fc-b269-e67f1c712206","Type":"ContainerStarted","Data":"27d6e79d2174f2821ac3ac9e8addb05cc25bd60808d86444d2f2bf58f1026c49"} Jul 12 22:58:03.894817 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.894750 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:03.894817 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.894772 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" event={"ID":"e733eb5d-1245-42fc-b269-e67f1c712206","Type":"ContainerStarted","Data":"e2294e23e0b7eb4c7b4ce7a0b4538f29c899a4f541c64e2d6a5cac1a090f9e06"} Jul 12 22:58:03.909586 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:03.909527 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" podStartSLOduration=1.909512804 podStartE2EDuration="1.909512804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:58:03.908261794 +0000 UTC m=+2865.776872618" watchObservedRunningTime="2026-07-12 22:58:03.909512804 +0000 UTC m=+2865.778123629" Jul 12 22:58:04.557761 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:04.557730 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wlkph_80c47a8b-861f-4216-98fc-ea06a6de52f7/dns/0.log" Jul 12 22:58:04.577363 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:04.577332 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wlkph_80c47a8b-861f-4216-98fc-ea06a6de52f7/kube-rbac-proxy/0.log" Jul 12 22:58:04.642307 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:04.642283 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vht46_84b7f93a-2be5-4673-8f1c-9a88435b4e91/dns-node-resolver/0.log" Jul 12 22:58:05.171951 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:05.171922 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7wm2n_255bf093-2970-45da-b373-4729e0e39673/node-ca/0.log" Jul 12 22:58:06.239321 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:06.239294 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747db99d8-78dk4_938ac42e-556c-483d-9306-cd0944e21be2/kube-auth-proxy/0.log" Jul 12 22:58:06.381660 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:06.381626 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-dsjgn_c82806ac-a7c2-4f98-9663-d8cacfd2e32a/payload-processing/0.log" Jul 12 22:58:07.143066 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:07.143023 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5bxkf_97597565-bdb0-469d-9f37-dc5018e77128/serve-healthcheck-canary/0.log" Jul 12 22:58:07.640327 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:07.640293 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-wtzwb_d3164201-3898-4439-907f-3f9d20dd5c57/insights-operator/0.log" Jul 12 22:58:07.641613 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:07.641590 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-wtzwb_d3164201-3898-4439-907f-3f9d20dd5c57/insights-operator/1.log" Jul 12 22:58:07.791472 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:07.791428 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zcx4l_9842646b-932e-4847-8502-d2ecbad5e029/kube-rbac-proxy/0.log" Jul 12 22:58:07.811266 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:07.811237 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zcx4l_9842646b-932e-4847-8502-d2ecbad5e029/exporter/0.log" Jul 12 22:58:07.832610 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:07.832532 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zcx4l_9842646b-932e-4847-8502-d2ecbad5e029/extractor/0.log" Jul 12 22:58:09.912534 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:09.912455 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xkcz8/perf-node-gather-daemonset-hdd4r" Jul 12 22:58:10.070897 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:10.070860 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d8865b84c-gjw92_fb8903d6-da56-485b-bbbe-98ebc583b80f/manager/0.log" Jul 12 22:58:10.185781 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:10.185700 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56d7986bbd-47jnj_33065002-dab1-48fc-9b08-a4eee34cd6b3/manager/0.log" Jul 12 22:58:10.205692 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:10.205664 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56d7986bbd-47jnj_33065002-dab1-48fc-9b08-a4eee34cd6b3/copy-manifests/0.log" Jul 12 22:58:11.571102 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:11.571078 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64b6b6c6c7-jt5zr_540c06e0-91a3-4174-ac0a-368ee41a8726/manager/0.log" Jul 12 22:58:13.276497 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:13.276471 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747db99d8-78dk4_938ac42e-556c-483d-9306-cd0944e21be2/kube-auth-proxy/0.log" Jul 12 22:58:13.382620 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:13.382580 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-dsjgn_c82806ac-a7c2-4f98-9663-d8cacfd2e32a/payload-processing/0.log" Jul 12 22:58:14.798013 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:14.797943 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5747db99d8-78dk4_938ac42e-556c-483d-9306-cd0944e21be2/kube-auth-proxy/0.log" Jul 12 22:58:14.910763 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:14.910217 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-dsjgn_c82806ac-a7c2-4f98-9663-d8cacfd2e32a/payload-processing/0.log" Jul 12 22:58:16.886050 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:16.886019 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-nblhr_47b707c0-46ba-4dab-8d0c-311ffc7d7e6e/migrator/0.log" Jul 12 22:58:16.906251 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:16.906215 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-nblhr_47b707c0-46ba-4dab-8d0c-311ffc7d7e6e/graceful-termination/0.log" Jul 12 22:58:18.395513 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.395481 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chlj5_b29b61f6-9dba-468c-adcd-146cbfc66a27/kube-multus-additional-cni-plugins/0.log" Jul 12 22:58:18.415051 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.415019 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chlj5_b29b61f6-9dba-468c-adcd-146cbfc66a27/egress-router-binary-copy/0.log" Jul 12 22:58:18.436588 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.436564 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chlj5_b29b61f6-9dba-468c-adcd-146cbfc66a27/cni-plugins/0.log" Jul 12 22:58:18.456436 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.456401 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chlj5_b29b61f6-9dba-468c-adcd-146cbfc66a27/bond-cni-plugin/0.log" Jul 12 22:58:18.475697 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.475672 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chlj5_b29b61f6-9dba-468c-adcd-146cbfc66a27/routeoverride-cni/0.log" Jul 12 22:58:18.495141 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.495113 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chlj5_b29b61f6-9dba-468c-adcd-146cbfc66a27/whereabouts-cni-bincopy/0.log" Jul 12 22:58:18.514096 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.514070 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chlj5_b29b61f6-9dba-468c-adcd-146cbfc66a27/whereabouts-cni/0.log" Jul 12 22:58:18.697855 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.697770 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qm5vb_1ed266b7-f4d9-4768-83e8-493ee6015789/kube-multus/0.log" Jul 12 22:58:18.782492 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.782464 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gq8dd_71466df1-3f21-4c6a-acbe-758aeea5531d/network-metrics-daemon/0.log" Jul 12 22:58:18.801089 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:18.801061 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gq8dd_71466df1-3f21-4c6a-acbe-758aeea5531d/kube-rbac-proxy/0.log" Jul 12 22:58:19.867784 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:19.867753 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-controller/0.log" Jul 12 22:58:19.883674 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:19.883639 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/1.log" Jul 12 22:58:19.883674 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:19.883647 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovn-acl-logging/0.log" Jul 12 22:58:19.902134 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:19.902096 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/kube-rbac-proxy-node/0.log" Jul 12 22:58:19.921814 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:19.921785 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/kube-rbac-proxy-ovn-metrics/0.log" Jul 12 22:58:19.942538 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:19.942506 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/northd/0.log" Jul 12 22:58:19.963172 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:19.963150 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/nbdb/0.log" Jul 12 22:58:19.982652 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:19.982627 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/sbdb/0.log" Jul 12 22:58:20.045702 ip-10-0-134-44 kubenswrapper[2563]: I0712 22:58:20.045629 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bvgbg_7bb73db5-1ffe-4b1e-9cbc-e6744f653d65/ovnkube-controller/0.log"