Jul 06 22:44:32.307347 ip-10-0-140-53 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 22:44:32.307362 ip-10-0-140-53 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 22:44:32.307369 ip-10-0-140-53 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 22:44:32.307606 ip-10-0-140-53 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 22:44:42.502823 ip-10-0-140-53 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 22:44:42.502844 ip-10-0-140-53 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9a33f94acdba49f99587c416eb24673d --
Jul 06 22:46:17.073836 ip-10-0-140-53 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 22:46:17.492861 ip-10-0-140-53 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 22:46:17.492861 ip-10-0-140-53 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 22:46:17.492861 ip-10-0-140-53 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 22:46:17.492861 ip-10-0-140-53 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 22:46:17.492861 ip-10-0-140-53 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 22:46:17.493602 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.492916    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 22:46:17.497976 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.497954    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:46:17.497976 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.497975    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:46:17.497976 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.497982    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.497986    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.497989    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.497992    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.497996    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.497999    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498002    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498004    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498007    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498010    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498013    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498016    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498023    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498027    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498030    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498033    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498035    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498038    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498041    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498043    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:46:17.498122 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498046    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498048    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498051    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498053    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498059    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498061    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498064    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498066    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498069    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498072    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498074    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498076    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498079    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498082    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498085    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498088    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498092    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498098    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498100    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498103    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:46:17.498609 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498106    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498108    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498111    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498113    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498116    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498118    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498121    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498123    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498126    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498131    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498134    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498136    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498139    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498141    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498144    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498147    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498149    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498152    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498155    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498157    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:46:17.499091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498160    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498163    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498168    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498170    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498173    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498176    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498179    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498182    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498185    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498187    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498190    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498193    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498197    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498200    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498205    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498207    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498210    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498212    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498215    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498217    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:46:17.499590 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498220    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498223    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498225    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498228    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498230    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498233    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498238    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498240    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498243    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498247    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498250    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498253    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498256    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498259    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498262    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498266    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.498269    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499450    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499459    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:46:17.500064 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499462    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499466    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499471    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499474    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499477    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499481    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499484    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499486    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499489    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499492    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499495    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499497    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499500    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499504    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499510    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499513    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499530    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499534    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499537    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:46:17.500535 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499540    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499543    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499545    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499548    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499551    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499555    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499557    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499562    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499565    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499568    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499570    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499573    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499577    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499580    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499583    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499586    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499588    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499591    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499594    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499597    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:46:17.501104 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499602    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499605    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499607    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499610    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499613    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499615    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499618    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499620    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499623    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499625    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499628    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499630    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499633    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499637    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499640    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499642    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499645    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499647    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499650    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:46:17.501608 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499653    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499655    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499658    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499660    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499663    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499666    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499671    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499675    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499679    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499682    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499685    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499688    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499692    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499695    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499697    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499700    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499703    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499705    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499708    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499713    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:46:17.502079 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499715    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499718    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499720    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499723    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499726    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499730    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499734    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499737    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499740    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499743    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499748    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499750    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499753    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499755    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499758    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499761    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499764    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499767    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.499770    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500418    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:46:17.502600 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500424    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500427    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500430    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500433    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500439    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500442    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500444    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500447    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500450    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500453    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500455    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500458    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500461    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500464    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500466    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500469    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500474    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500476    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500479    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500481    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:46:17.503089 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500484    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500487    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500490    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500492    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500495    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500497    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500500    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500502    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500505    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500512    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500514    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500531    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500534    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500537    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500540    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500543    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500545    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500555    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500557    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:46:17.503594 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500560    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500564    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500570    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500574    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500577    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500579    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500582    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500585    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500587    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500590    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500593    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500595    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500597    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500600    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500605    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500607    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500610    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500612    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500615    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500618    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:46:17.504160 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500620    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500623    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500626    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500628    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500631    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500633    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500638    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500641    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500643    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500646    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500648    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500656    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500658    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500661    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500663    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500666    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500668    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500671    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500674    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500678    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:46:17.504656 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500681    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500683    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500686    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500688    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500691    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500693    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500697    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500701    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500705    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500708    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500711    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500716    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500719    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500721    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500727    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500731    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500734    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500737    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.500740    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:46:17.505147 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.500978    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.500987    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.500994    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.500998    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501002    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501005    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501009    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501013    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501017    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501020    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501023    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501026    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501029    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501032    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501035    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501038    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501040    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501043    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501048    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501051    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501056    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501059    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501064    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501069    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 22:46:17.505662 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501072    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501075    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501080    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501087    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501090    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501094    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501097    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501100    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501105    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501108    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501114    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501116    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501119    2563 flags.go:64] FLAG: --enable-server="true"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501123    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501127    2563 flags.go:64] FLAG: --event-burst="100"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501130    2563 flags.go:64] FLAG: --event-qps="50"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501133    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501136    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501138    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501142    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501144    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501147    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501150    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501153    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501155    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 22:46:17.506235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501159    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501162    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501165    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501168    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501171    2563 flags.go:64] FLAG: --feature-gates=""
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501174    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501177    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501180    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501183    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501187    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501192    2563 flags.go:64] FLAG: --help="false"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501195    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501198    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501201    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501204    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501207    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501211    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501214    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501217    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501219    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501222    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501224    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501227    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501230    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 06 22:46:17.506873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501233    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501235    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501238    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501240    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501243    2563 flags.go:64] FLAG: --lock-file=""
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501246    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501249    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501251    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501259    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501262    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501265    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501268    2563 flags.go:64] FLAG: --logging-format="text"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501270    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501274    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501276    2563 flags.go:64] FLAG: --manifest-url=""
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501279    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501283    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501288    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501292    2563 flags.go:64] FLAG: --max-pods="110"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501295    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501298    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501300    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501304    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501306    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501309    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 22:46:17.507470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501312    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501320    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501323    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501326    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501328    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501331    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501336    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501339    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501342    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501344    2563 flags.go:64] FLAG: --port="10250"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501347    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501350    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f92d112bfde3e9fe"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501353    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501356    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501359    2563 flags.go:64] FLAG: --register-node="true"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501362    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501365    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501368    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501371    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501373    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501377    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501380    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501383    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501385    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501389    2563 flags.go:64] FLAG: --runonce="false"
Jul 06 22:46:17.508099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501392    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501396    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501398    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501401    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501404    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501406    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501409    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501412    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501415    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501417    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501420    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501423    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501425    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501428    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501430    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501436    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501439    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501441    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501446    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501449    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501451    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501454    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501457    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501459    2563 flags.go:64] FLAG: --v="2"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501463    2563 flags.go:64] FLAG: --version="false"
Jul 06 22:46:17.508772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501467    2563 flags.go:64] FLAG: --vmodule=""
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501471    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.501474    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501617    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501621    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501624    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501628    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501630    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501634    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501637    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501639    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501642    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501645    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501647    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501651    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501655    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501658    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501660    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501663    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:46:17.509400 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501665    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501668    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501671    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501673    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501676    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501678    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501681    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501684    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501686    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501689    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501692    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501695    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501697    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501700    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501703    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501705    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501708    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501710    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501713    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501716    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:46:17.509888 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501719    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501722    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501725    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501727    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501729    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501732    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501734    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501737    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501741    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501744    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501747    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501750    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501752    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501756    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501759    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501762    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501764    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501767    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501769    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:46:17.510367 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501771    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501774    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501776    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501779    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501781    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501784    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501786    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501788    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501792    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501794    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501797    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501799    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501803    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501805    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501808    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501811    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501813    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501816    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501819    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501821    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:46:17.510846 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501824    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501826    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501829    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501831    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501833    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501835    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501838    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501840    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501843    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501845    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501847    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501850    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501852    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501855    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501857    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501859    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501861    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501864    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501866    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501869    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:46:17.511342 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501871    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501874    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501876    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.501879    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.502618    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.509925    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.509943    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.509964    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.509970    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510036    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510042    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510045    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510048    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510051    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510055    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:46:17.511832 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510059    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510062    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510065    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510068    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510071    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510073    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510076    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510078    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510081    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510084    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510086    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510089    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510092    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510094    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510097    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510099    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510102    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510104    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510106    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510109    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:46:17.512202 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510111    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510114    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510116    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510121    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510123    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510126    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510128    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510131    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510133    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510136    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510138    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510141    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510143    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510146    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510148    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510151    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510153    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510156    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510158    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510161    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:46:17.512697 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510164    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510166    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510169    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510171    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510174    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510176    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510178    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510181    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510184    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510186    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510189    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510193    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510196    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510199    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510202    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510204    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510208    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510211    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510213    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:46:17.513166 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510215    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510218    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510220    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510223    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510225    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510227    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510230    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510233    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510235    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510238    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510240    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510243    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510245    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510248    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510250    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510253    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510255    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510258    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510260    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:46:17.513640 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510262    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510265    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510268    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510270    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510273    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510276    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510278    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510281    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510285    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510289    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510291    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510295    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510297    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510301    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510303    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:46:17.514098 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.510309    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510445    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510449    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510453    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510456    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510458    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510461    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510463    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510466    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510468    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510471    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510474    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510476    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510478    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510481    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510483    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510486    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510488    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510491    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510493    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:46:17.514467 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510496    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510498    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510501    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510504    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510506    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510509    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510511    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510514    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510516    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510544    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510547    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510549    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510552    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510554    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510557    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510559    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510562    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510564    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510567    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510569    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:46:17.515132 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510572    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510575    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510577    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510580    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510582    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510586    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510590    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510593    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510597    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510600    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510602    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510605    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510608    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510612    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510615    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510618    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510621    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510624    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510627    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:46:17.515641 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510630    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510632    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510635    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510638    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510640    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510643    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510645    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510648    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510651    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510653    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510656    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510658    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510660    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510663    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510665    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510668    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510670    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510672    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510675    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510677    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:46:17.516091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510680    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510682    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510684    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510687    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510689    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510692    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510694    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510696    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510699    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510701    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510706    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510708    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510711    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510713    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510716    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510718    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510721    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510723    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510725    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510728    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:46:17.516607 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:17.510731    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:46:17.517081 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.510736    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 22:46:17.517081 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.510856    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 22:46:17.517081 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.514583    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 22:46:17.517081 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.516248    2563 server.go:1019] "Starting client certificate rotation"
Jul 06 22:46:17.517081 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.516363    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 22:46:17.517081 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.516410    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 22:46:17.519291 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.519274    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 22:46:17.522510 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.522491    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 22:46:17.534948 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.534926    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 06 22:46:17.537388 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.537375    2563 log.go:25] "Validated CRI v1 image API"
Jul 06 22:46:17.538166 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.538149    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 22:46:17.542556 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.542513    2563 fs.go:135] Filesystem UUIDs: map[59b7067e-199e-46d3-affa-e2f546444789:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9b0fc085-8c74-4582-968c-10c6f7248c95:/dev/nvme0n1p4]
Jul 06 22:46:17.542620 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.542555    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 22:46:17.548013 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.547994    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 22:46:17.548963 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.548824    2563 manager.go:216] Machine: {Timestamp:2026-07-06 22:46:17.546344809 +0000 UTC m=+0.354792273 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3065990 MemoryCapacity:32812183552 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29b09a9fcefddc205909dfcf0f96b4 SystemUUID:ec29b09a-9fce-fddc-2059-09dfcf0f96b4 BootID:9a33f94a-cdba-49f9-9587-c416eb24673d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406093824 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4a:80:d4:c6:8b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4a:80:d4:c6:8b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:23:f3:e2:a7:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812183552 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 22:46:17.548963 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.548964    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 22:46:17.549072 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.549060    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 22:46:17.552466 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.552441    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 22:46:17.552633 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.552468    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-53.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 22:46:17.553257 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.553246    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 22:46:17.553289 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.553259    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 22:46:17.553289 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.553267    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 22:46:17.554056 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.554045    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 22:46:17.555822 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.555810    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 22:46:17.559205 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.559192    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 22:46:17.559326 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.559316    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 22:46:17.559371 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.559349    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 22:46:17.559371 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.559366    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 22:46:17.559443 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.559380    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 22:46:17.559443 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.559390    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 06 22:46:17.559443 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.559399    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 22:46:17.559767 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.559756    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 22:46:17.559809 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.559774    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 22:46:17.571045 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.571026    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 22:46:17.571551 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.571512    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-53.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 22:46:17.571724 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.571707    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 22:46:17.573574 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.573559    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 22:46:17.573614 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.573592    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 22:46:17.575073 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575061    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 22:46:17.575130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575077    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 22:46:17.575130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575092    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 22:46:17.575130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575098    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 22:46:17.575130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575107    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 22:46:17.575130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575113    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 22:46:17.575130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575122    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 22:46:17.575130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575128    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 22:46:17.575130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575134    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 22:46:17.575337 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575139    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 22:46:17.575337 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575168    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 22:46:17.575337 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.575176    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 22:46:17.576105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.576094    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 22:46:17.576105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.576104    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 22:46:17.578128 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.578107    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-53.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 22:46:17.579384 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.579371    2563 server.go:1268] "Started kubelet"
Jul 06 22:46:17.579479 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.579445    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 22:46:17.579908 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.579776    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 22:46:17.580054 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.580040    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 22:46:17.581295 ip-10-0-140-53 systemd[1]: Started Kubernetes Kubelet.
Jul 06 22:46:17.582275 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.582255    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 22:46:17.582770 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.582757    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 22:46:17.587658 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.587636    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 22:46:17.591582 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.591556    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 22:46:17.591582 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.591573    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 22:46:17.591717 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.591656    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:17.592235 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.591945    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vlq88"
Jul 06 22:46:17.592333 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592295    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 22:46:17.592333 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592304    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 22:46:17.592333 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592322    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 22:46:17.592591 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592550    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 22:46:17.592591 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592565    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 22:46:17.592591 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592614    2563 factory.go:55] Registering systemd factory
Jul 06 22:46:17.592591 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592632    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 06 22:46:17.592884 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592832    2563 factory.go:153] Registering CRI-O factory
Jul 06 22:46:17.592884 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592849    2563 factory.go:223] Registration of the crio container factory successfully
Jul 06 22:46:17.592970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592898    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 22:46:17.592970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592920    2563 factory.go:103] Registering Raw factory
Jul 06 22:46:17.592970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.592933    2563 manager.go:1196] Started watching for new ooms in manager
Jul 06 22:46:17.593625 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.593606    2563 manager.go:349] Starting recovery of all containers
Jul 06 22:46:17.593710 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.592193    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-53.ec2.internal.18bfd4d265df04db  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-53.ec2.internal,UID:ip-10-0-140-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-53.ec2.internal,},FirstTimestamp:2026-07-06 22:46:17.579341019 +0000 UTC m=+0.387788522,LastTimestamp:2026-07-06 22:46:17.579341019 +0000 UTC m=+0.387788522,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-53.ec2.internal,}"
Jul 06 22:46:17.599630 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.599608    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vlq88"
Jul 06 22:46:17.600976 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.600740    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 22:46:17.600976 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.600946    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 22:46:17.601705 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.600853    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-53.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 22:46:17.605261 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.605240    2563 manager.go:354] Recovery completed
Jul 06 22:46:17.609273 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.609257    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:46:17.611499 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.611481    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:46:17.611593 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.611536    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:46:17.611593 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.611553    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:46:17.612122 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.612110    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 22:46:17.612122 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.612123    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 22:46:17.612222 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.612138    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 22:46:17.612222 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.612172    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:46:17.613205 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.613192    2563 policy_none.go:49] "None policy: Start"
Jul 06 22:46:17.613256 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.613208    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 22:46:17.613256 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.613218    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 22:46:17.613363 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.613344    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:46:17.613397 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.613377    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:46:17.613397 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.613387    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:46:17.614278 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.614268    2563 policy_none.go:47] "Start"
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.655112    2563 manager.go:337] "Starting Device Plugin manager"
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.655146    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.655163    2563 server.go:87] "Starting device plugin registration server"
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.655468    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.655482    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.655597    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.655692    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.655702    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.656219    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 22:46:17.663105 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.656275    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:17.690707 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.690675    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 22:46:17.690841 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.690724    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 22:46:17.690841 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.690746    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 22:46:17.690841 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.690783    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 22:46:17.692894 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.692876    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:46:17.755991 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.755893    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:46:17.757051 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.757034    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:46:17.757128 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.757067    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:46:17.757128 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.757077    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:46:17.757128 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.757101    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.765581 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.765565    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.765643 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.765588    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-53.ec2.internal\": node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:17.790907 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.790875    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:17.790907 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.790904    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal"]
Jul 06 22:46:17.791092 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.790983    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:46:17.791892 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.791877    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:46:17.791976 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.791907    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:46:17.791976 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.791917    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:46:17.793222 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.793209    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:46:17.793390 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.793376    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.793465 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.793404    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:46:17.793955 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.793938    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:46:17.794039 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.793970    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:46:17.794039 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.793985    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:46:17.794039 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.793986    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:46:17.794039 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.794012    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:46:17.794039 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.794028    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:46:17.795330 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.795313    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.795414 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.795344    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:46:17.796024 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.796003    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:46:17.796102 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.796036    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:46:17.796102 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.796047    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:46:17.815193 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.815158    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-53.ec2.internal\" not found" node="ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.818893 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.818875    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-53.ec2.internal\" not found" node="ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.891941 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.891911    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:17.893066 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.893050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/058ae59d5825d565944e6642e3273d0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal\" (UID: \"058ae59d5825d565944e6642e3273d0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.893125 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.893074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/058ae59d5825d565944e6642e3273d0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal\" (UID: \"058ae59d5825d565944e6642e3273d0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.992266 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:17.992219    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:17.993410 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.993394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/058ae59d5825d565944e6642e3273d0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal\" (UID: \"058ae59d5825d565944e6642e3273d0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.993492 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.993424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/058ae59d5825d565944e6642e3273d0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal\" (UID: \"058ae59d5825d565944e6642e3273d0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.993492 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.993454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3383faa9d76785b568517c84a7fb072b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-53.ec2.internal\" (UID: \"3383faa9d76785b568517c84a7fb072b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.993609 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.993514    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/058ae59d5825d565944e6642e3273d0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal\" (UID: \"058ae59d5825d565944e6642e3273d0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:17.993609 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:17.993516    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/058ae59d5825d565944e6642e3273d0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal\" (UID: \"058ae59d5825d565944e6642e3273d0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:18.092960 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:18.092897    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:18.094016 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.093997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3383faa9d76785b568517c84a7fb072b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-53.ec2.internal\" (UID: \"3383faa9d76785b568517c84a7fb072b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:18.094090 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.094058    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3383faa9d76785b568517c84a7fb072b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-53.ec2.internal\" (UID: \"3383faa9d76785b568517c84a7fb072b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:18.118112 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.118084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:18.123781 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.123749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:18.193601 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:18.193561    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:18.293997 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:18.293968    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:18.394544 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:18.394455    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:18.495084 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:18.495040    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:18.515322 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.515292    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 22:46:18.515495 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.515472    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 22:46:18.595609 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:18.595571    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:18.597608 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.597590    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:46:18.601832 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.601803    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 22:41:18 +0000 UTC" deadline="2028-01-15 18:11:19.491034316 +0000 UTC"
Jul 06 22:46:18.601885 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.601833    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13387h25m0.889203778s"
Jul 06 22:46:18.695942 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:18.695867    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-53.ec2.internal\" not found"
Jul 06 22:46:18.751828 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.751803    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:46:18.788750 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:18.788586    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod058ae59d5825d565944e6642e3273d0e.slice/crio-4772b7940bf2a413da2ad7f8df4fe50bdc0e480479ca227b6da2e43d7be81965 WatchSource:0}: Error finding container 4772b7940bf2a413da2ad7f8df4fe50bdc0e480479ca227b6da2e43d7be81965: Status 404 returned error can't find the container with id 4772b7940bf2a413da2ad7f8df4fe50bdc0e480479ca227b6da2e43d7be81965
Jul 06 22:46:18.791055 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:18.791031    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3383faa9d76785b568517c84a7fb072b.slice/crio-8eae7fafa8b5963bfb44bf39a29b04d99f81cf142f6a17229e840e990370fe1a WatchSource:0}: Error finding container 8eae7fafa8b5963bfb44bf39a29b04d99f81cf142f6a17229e840e990370fe1a: Status 404 returned error can't find the container with id 8eae7fafa8b5963bfb44bf39a29b04d99f81cf142f6a17229e840e990370fe1a
Jul 06 22:46:18.791771 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.791754    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:18.792789 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.792774    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:46:18.804949 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.804929    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 22:46:18.806790 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.806771    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal"
Jul 06 22:46:18.812785 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.812763    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 22:46:18.870433 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:18.870401    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:46:20.825893 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.825792    2563 apiserver.go:52] "Watching apiserver"
Jul 06 22:46:20.836011 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.835978    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 22:46:20.837259 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.836868    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-t7t9f","kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal","openshift-cluster-node-tuning-operator/tuned-bbfcc","openshift-image-registry/node-ca-qhn45","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal","openshift-multus/multus-additional-cni-plugins-mk4bk","openshift-multus/network-metrics-daemon-4jvqw","openshift-network-operator/iptables-alerter-cqtrv","kube-system/global-pull-secret-syncer-mgsdw","kube-system/konnectivity-agent-7nkqp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82","openshift-dns/node-resolver-n6jp9","openshift-multus/multus-pmslq","openshift-network-diagnostics/network-check-target-72np5"]
Jul 06 22:46:20.838416 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.838322    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:20.838514 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.838411    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:20.839463 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.839438    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.840706 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.840680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.842000 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.841980    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.842861 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.842840    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 22:46:20.843100 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.843081    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 22:46:20.843182 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.843122    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 22:46:20.843274 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.843083    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 22:46:20.843341 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.843285    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t96cf\""
Jul 06 22:46:20.843405 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.843386    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:20.843482 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.843449    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:20.843541 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.843481    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kth7n\""
Jul 06 22:46:20.843541 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.843534    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 22:46:20.844858 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.844242    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 22:46:20.844858 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.844312    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kjn6w\""
Jul 06 22:46:20.844858 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.844245    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.844858 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.844492    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 22:46:20.845153 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.845131    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 22:46:20.845684 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.845668    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:20.845777 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.845737    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:20.846674 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.846657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.846786 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.846764    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 22:46:20.847054 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.847035    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9fkc5\""
Jul 06 22:46:20.848096 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.848080    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.849852 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.849832    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 22:46:20.849852 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.849847    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 22:46:20.849987 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.849977    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:20.850085 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.850065    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 22:46:20.850394 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.850374    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 22:46:20.850484 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.850463    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7www5\""
Jul 06 22:46:20.850612 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.850598    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 22:46:20.850686 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.850605    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 22:46:20.850722 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.850702    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 22:46:20.850865 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.850848    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 22:46:20.850936 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.850921    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pc5sr\""
Jul 06 22:46:20.851035 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.851017    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.852309 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.852293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.852777 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.852728    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 22:46:20.852867 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.852779    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 22:46:20.852929 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.852913    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-86knk\""
Jul 06 22:46:20.853341 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.853323    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 22:46:20.853414 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.853394    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 22:46:20.853467 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.853429    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2z7cj\""
Jul 06 22:46:20.853619 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.853602    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 22:46:20.853690 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.853622    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 22:46:20.853924 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.853906    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 22:46:20.854774 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.854758    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 22:46:20.855440 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.855184    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f7t4d\""
Jul 06 22:46:20.855440 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.855198    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 22:46:20.855440 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.855233    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 22:46:20.858665 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.858630    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 22:46:20.859773 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.859753    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 22:46:20.867230 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867125    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-sys\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.867230 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867158    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-lib-modules\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.867230 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.867433 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-socket-dir-parent\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.867433 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867283    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa35364e-53e3-4b89-a7bf-584d76ce3005-hosts-file\") pod \"node-resolver-n6jp9\" (UID: \"fa35364e-53e3-4b89-a7bf-584d76ce3005\") " pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.867433 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-var-lib-cni-multus\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.867433 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-systemd\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.867433 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-system-cni-dir\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.867573 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-sys-fs\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.867573 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-run-k8s-cni-cncf-io\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.867573 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/493fe052-d0fd-4f5d-870b-757109e4dd9d-tmp\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.867573 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7jgw\" (UniqueName: \"kubernetes.io/projected/493fe052-d0fd-4f5d-870b-757109e4dd9d-kube-api-access-d7jgw\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.867730 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79g74\" (UniqueName: \"kubernetes.io/projected/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-kube-api-access-79g74\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.867730 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-run-systemd\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.867730 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-sysctl-d\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.867730 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-sysctl-conf\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.867730 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867697    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.867730 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-socket-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.867942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867737    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-cni-binary-copy\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.867942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-etc-kubernetes\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.867942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-etc-openvswitch\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.867942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-542w4\" (UniqueName: \"kubernetes.io/projected/fa35364e-53e3-4b89-a7bf-584d76ce3005-kube-api-access-542w4\") pod \"node-resolver-n6jp9\" (UID: \"fa35364e-53e3-4b89-a7bf-584d76ce3005\") " pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.867942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-host\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.867942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-os-release\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.867942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76cf142e-ee9d-4b89-9f27-9233e493ffad-iptables-alerter-script\") pod \"iptables-alerter-cqtrv\" (UID: \"76cf142e-ee9d-4b89-9f27-9233e493ffad\") " pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.867942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-var-lib-kubelet\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68774\" (UniqueName: \"kubernetes.io/projected/76cf142e-ee9d-4b89-9f27-9233e493ffad-kube-api-access-68774\") pod \"iptables-alerter-cqtrv\" (UID: \"76cf142e-ee9d-4b89-9f27-9233e493ffad\") " pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-registration-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.867994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-var-lib-cni-bin\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-hostroot\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-run-multus-certs\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-run-ovn\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868101    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-log-socket\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868141    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-cni-binary-copy\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.868223 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5403f15-1e81-4b5d-b409-6b18e29e201f-agent-certs\") pod \"konnectivity-agent-7nkqp\" (UID: \"b5403f15-1e81-4b5d-b409-6b18e29e201f\") " pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-kubelet-config\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-cnibin\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-os-release\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92dvm\" (UniqueName: \"kubernetes.io/projected/b05d5380-0b82-4ade-9481-2dcd94f82354-kube-api-access-92dvm\") pod \"node-ca-qhn45\" (UID: \"b05d5380-0b82-4ade-9481-2dcd94f82354\") " pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5403f15-1e81-4b5d-b409-6b18e29e201f-konnectivity-ca\") pod \"konnectivity-agent-7nkqp\" (UID: \"b5403f15-1e81-4b5d-b409-6b18e29e201f\") " pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868442    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-run-netns\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-run-netns\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-var-lib-kubelet\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-device-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-system-cni-dir\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.868606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-cni-netd\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30f986f8-0422-4496-af7c-42790ddb5843-ovnkube-config\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-sysconfig\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-kubernetes\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-etc-selinux\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-cni-dir\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-tuned\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-cnibin\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868878    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868894    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdzw4\" (UniqueName: \"kubernetes.io/projected/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-kube-api-access-sdzw4\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-run-openvswitch\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.868996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqssk\" (UniqueName: \"kubernetes.io/projected/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-kube-api-access-sqssk\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-kubelet\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869068 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-run\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869076    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b05d5380-0b82-4ade-9481-2dcd94f82354-serviceca\") pod \"node-ca-qhn45\" (UID: \"b05d5380-0b82-4ade-9481-2dcd94f82354\") " pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-systemd-units\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-slash\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-var-lib-openvswitch\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30f986f8-0422-4496-af7c-42790ddb5843-ovn-node-metrics-cert\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869201    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6rbt\" (UniqueName: \"kubernetes.io/projected/30f986f8-0422-4496-af7c-42790ddb5843-kube-api-access-w6rbt\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa35364e-53e3-4b89-a7bf-584d76ce3005-tmp-dir\") pod \"node-resolver-n6jp9\" (UID: \"fa35364e-53e3-4b89-a7bf-584d76ce3005\") " pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b05d5380-0b82-4ade-9481-2dcd94f82354-host\") pod \"node-ca-qhn45\" (UID: \"b05d5380-0b82-4ade-9481-2dcd94f82354\") " pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76cf142e-ee9d-4b89-9f27-9233e493ffad-host-slash\") pod \"iptables-alerter-cqtrv\" (UID: \"76cf142e-ee9d-4b89-9f27-9233e493ffad\") " pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-dbus\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-conf-dir\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-daemon-config\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-cni-bin\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m5w9\" (UniqueName: \"kubernetes.io/projected/b8a3215b-b208-420a-82f1-4907c79efac1-kube-api-access-4m5w9\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:20.869791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:20.870323 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869511    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-node-log\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.870323 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30f986f8-0422-4496-af7c-42790ddb5843-ovnkube-script-lib\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.870323 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-modprobe-d\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.870323 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-run-ovn-kubernetes\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.870323 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.869652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30f986f8-0422-4496-af7c-42790ddb5843-env-overrides\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.891963 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.891935    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hmq7l"
Jul 06 22:46:20.898367 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.898341    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hmq7l"
Jul 06 22:46:20.962711 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.962632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal" event={"ID":"3383faa9d76785b568517c84a7fb072b","Type":"ContainerStarted","Data":"8eae7fafa8b5963bfb44bf39a29b04d99f81cf142f6a17229e840e990370fe1a"}
Jul 06 22:46:20.964056 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.964019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal" event={"ID":"058ae59d5825d565944e6642e3273d0e","Type":"ContainerStarted","Data":"4772b7940bf2a413da2ad7f8df4fe50bdc0e480479ca227b6da2e43d7be81965"}
Jul 06 22:46:20.970324 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970293    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:46:20.970470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.970470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-tuned\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.970470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970425    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-cnibin\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.970470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdzw4\" (UniqueName: \"kubernetes.io/projected/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-kube-api-access-sdzw4\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.970700 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:20.970700 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-run-openvswitch\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.970700 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqssk\" (UniqueName: \"kubernetes.io/projected/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-kube-api-access-sqssk\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.970700 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-kubelet\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.970700 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-run\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.970700 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970667    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b05d5380-0b82-4ade-9481-2dcd94f82354-serviceca\") pod \"node-ca-qhn45\" (UID: \"b05d5380-0b82-4ade-9481-2dcd94f82354\") " pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.970700 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-systemd-units\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971025 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-slash\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971025 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-var-lib-openvswitch\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971025 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30f986f8-0422-4496-af7c-42790ddb5843-ovn-node-metrics-cert\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971025 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6rbt\" (UniqueName: \"kubernetes.io/projected/30f986f8-0422-4496-af7c-42790ddb5843-kube-api-access-w6rbt\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971025 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa35364e-53e3-4b89-a7bf-584d76ce3005-tmp-dir\") pod \"node-resolver-n6jp9\" (UID: \"fa35364e-53e3-4b89-a7bf-584d76ce3005\") " pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.971025 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970862    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b05d5380-0b82-4ade-9481-2dcd94f82354-host\") pod \"node-ca-qhn45\" (UID: \"b05d5380-0b82-4ade-9481-2dcd94f82354\") " pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.971025 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76cf142e-ee9d-4b89-9f27-9233e493ffad-host-slash\") pod \"iptables-alerter-cqtrv\" (UID: \"76cf142e-ee9d-4b89-9f27-9233e493ffad\") " pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.971025 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.970934    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.971025 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-dbus\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-conf-dir\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-daemon-config\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-cni-bin\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m5w9\" (UniqueName: \"kubernetes.io/projected/b8a3215b-b208-420a-82f1-4907c79efac1-kube-api-access-4m5w9\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971210    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b05d5380-0b82-4ade-9481-2dcd94f82354-serviceca\") pod \"node-ca-qhn45\" (UID: \"b05d5380-0b82-4ade-9481-2dcd94f82354\") " pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971253    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-cnibin\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-node-log\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30f986f8-0422-4496-af7c-42790ddb5843-ovnkube-script-lib\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-modprobe-d\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-run-ovn-kubernetes\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30f986f8-0422-4496-af7c-42790ddb5843-env-overrides\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-sys\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.971442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-lib-modules\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-socket-dir-parent\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa35364e-53e3-4b89-a7bf-584d76ce3005-hosts-file\") pod \"node-resolver-n6jp9\" (UID: \"fa35364e-53e3-4b89-a7bf-584d76ce3005\") " pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-var-lib-cni-multus\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-systemd\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971615    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-system-cni-dir\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-sys-fs\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-run-k8s-cni-cncf-io\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/493fe052-d0fd-4f5d-870b-757109e4dd9d-tmp\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7jgw\" (UniqueName: \"kubernetes.io/projected/493fe052-d0fd-4f5d-870b-757109e4dd9d-kube-api-access-d7jgw\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79g74\" (UniqueName: \"kubernetes.io/projected/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-kube-api-access-79g74\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-run-systemd\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-sysctl-d\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-sysctl-conf\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971916    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.972209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-socket-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.971988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-cni-binary-copy\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-etc-kubernetes\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-etc-openvswitch\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-542w4\" (UniqueName: \"kubernetes.io/projected/fa35364e-53e3-4b89-a7bf-584d76ce3005-kube-api-access-542w4\") pod \"node-resolver-n6jp9\" (UID: \"fa35364e-53e3-4b89-a7bf-584d76ce3005\") " pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-host\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-os-release\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76cf142e-ee9d-4b89-9f27-9233e493ffad-iptables-alerter-script\") pod \"iptables-alerter-cqtrv\" (UID: \"76cf142e-ee9d-4b89-9f27-9233e493ffad\") " pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-var-lib-kubelet\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68774\" (UniqueName: \"kubernetes.io/projected/76cf142e-ee9d-4b89-9f27-9233e493ffad-kube-api-access-68774\") pod \"iptables-alerter-cqtrv\" (UID: \"76cf142e-ee9d-4b89-9f27-9233e493ffad\") " pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-registration-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-var-lib-cni-bin\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-hostroot\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-run-multus-certs\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.972332    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-run-ovn\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-log-socket\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.973186 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.972407    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret podName:d7e29fc5-f57a-45bd-b114-f48b56ea7b6c nodeName:}" failed. No retries permitted until 2026-07-06 22:46:21.472389371 +0000 UTC m=+3.014720550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret") pod "global-pull-secret-syncer-mgsdw" (UID: "d7e29fc5-f57a-45bd-b114-f48b56ea7b6c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-cni-binary-copy\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972479    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-systemd-units\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.972553    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.972586    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs podName:b8a3215b-b208-420a-82f1-4907c79efac1 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:21.47257324 +0000 UTC m=+3.014904409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs") pod "network-metrics-daemon-4jvqw" (UID: "b8a3215b-b208-420a-82f1-4907c79efac1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972443    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-log-socket\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972884    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-systemd\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972938    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-slash\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.972982    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-var-lib-openvswitch\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.973113    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-cni-binary-copy\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.973176    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-run-openvswitch\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.973226    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-system-cni-dir\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.973310    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-sys-fs\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.973970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.973358    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-run-k8s-cni-cncf-io\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.974504    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-sys\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.974693    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-node-log\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975035    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa35364e-53e3-4b89-a7bf-584d76ce3005-tmp-dir\") pod \"node-resolver-n6jp9\" (UID: \"fa35364e-53e3-4b89-a7bf-584d76ce3005\") " pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975104    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30f986f8-0422-4496-af7c-42790ddb5843-ovnkube-script-lib\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975117    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b05d5380-0b82-4ade-9481-2dcd94f82354-host\") pod \"node-ca-qhn45\" (UID: \"b05d5380-0b82-4ade-9481-2dcd94f82354\") " pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975181    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-modprobe-d\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975192    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76cf142e-ee9d-4b89-9f27-9233e493ffad-host-slash\") pod \"iptables-alerter-cqtrv\" (UID: \"76cf142e-ee9d-4b89-9f27-9233e493ffad\") " pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975210    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-run-ovn-kubernetes\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975265    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975374    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-kubelet\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975492    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-dbus\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975496    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30f986f8-0422-4496-af7c-42790ddb5843-env-overrides\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975545    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-conf-dir\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975569    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-run-systemd\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975590    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-tuned\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975598    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-run\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975693    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-sysctl-d\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.976457 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975767    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-sysctl-conf\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975778    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-etc-kubernetes\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975841    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975891    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-etc-openvswitch\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975944    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/493fe052-d0fd-4f5d-870b-757109e4dd9d-tmp\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976014    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-var-lib-kubelet\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976046    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-socket-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.975622    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-cni-bin\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976088    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-daemon-config\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976123    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-os-release\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976136    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-host\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976221    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-cni-binary-copy\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976284    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-run-multus-certs\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976293    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76cf142e-ee9d-4b89-9f27-9233e493ffad-iptables-alerter-script\") pod \"iptables-alerter-cqtrv\" (UID: \"76cf142e-ee9d-4b89-9f27-9233e493ffad\") " pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976295    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-registration-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976329    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-var-lib-cni-bin\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976346    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-socket-dir-parent\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976358    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-hostroot\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.977595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976362    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-run-ovn\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976399    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-var-lib-cni-multus\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976420    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa35364e-53e3-4b89-a7bf-584d76ce3005-hosts-file\") pod \"node-resolver-n6jp9\" (UID: \"fa35364e-53e3-4b89-a7bf-584d76ce3005\") " pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976451    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5403f15-1e81-4b5d-b409-6b18e29e201f-agent-certs\") pod \"konnectivity-agent-7nkqp\" (UID: \"b5403f15-1e81-4b5d-b409-6b18e29e201f\") " pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976459    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-lib-modules\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976479    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-kubelet-config\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976507    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-kubelet-config\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-cnibin\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-os-release\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976550    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-cnibin\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92dvm\" (UniqueName: \"kubernetes.io/projected/b05d5380-0b82-4ade-9481-2dcd94f82354-kube-api-access-92dvm\") pod \"node-ca-qhn45\" (UID: \"b05d5380-0b82-4ade-9481-2dcd94f82354\") " pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976593    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5403f15-1e81-4b5d-b409-6b18e29e201f-konnectivity-ca\") pod \"konnectivity-agent-7nkqp\" (UID: \"b5403f15-1e81-4b5d-b409-6b18e29e201f\") " pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976627    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-os-release\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-run-netns\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976680    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-run-netns\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-run-netns\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976710    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-host-run-netns\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.978550 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-var-lib-kubelet\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-device-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-system-cni-dir\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976841    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-var-lib-kubelet\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-cni-netd\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30f986f8-0422-4496-af7c-42790ddb5843-ovnkube-config\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976904    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-device-dir\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976903    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-system-cni-dir\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-sysconfig\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-kubernetes\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-etc-selinux\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976995    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-sysconfig\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.977027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-cni-dir\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.977091    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-etc-selinux\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.977087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.977136    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.977146    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5403f15-1e81-4b5d-b409-6b18e29e201f-konnectivity-ca\") pod \"konnectivity-agent-7nkqp\" (UID: \"b5403f15-1e81-4b5d-b409-6b18e29e201f\") " pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:20.979478 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.976958    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30f986f8-0422-4496-af7c-42790ddb5843-host-cni-netd\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.980511 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.977217    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/493fe052-d0fd-4f5d-870b-757109e4dd9d-etc-kubernetes\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.980511 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.977289    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-multus-cni-dir\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.980511 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.977616    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30f986f8-0422-4496-af7c-42790ddb5843-ovnkube-config\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.980511 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.977899    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30f986f8-0422-4496-af7c-42790ddb5843-ovn-node-metrics-cert\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.980511 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.980106    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5403f15-1e81-4b5d-b409-6b18e29e201f-agent-certs\") pod \"konnectivity-agent-7nkqp\" (UID: \"b5403f15-1e81-4b5d-b409-6b18e29e201f\") " pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:20.980797 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.980585    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:46:20.980797 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.980606    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:46:20.980797 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.980616    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8cqjz for pod openshift-network-diagnostics/network-check-target-72np5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:20.980797 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:20.980716    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz podName:5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b nodeName:}" failed. No retries permitted until 2026-07-06 22:46:21.480695134 +0000 UTC m=+3.023026283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8cqjz" (UniqueName: "kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz") pod "network-check-target-72np5" (UID: "5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:20.981982 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.981960    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqssk\" (UniqueName: \"kubernetes.io/projected/bba4f2f3-ca52-4cb3-a21d-830bfc6815c2-kube-api-access-sqssk\") pod \"multus-pmslq\" (UID: \"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2\") " pod="openshift-multus/multus-pmslq"
Jul 06 22:46:20.982370 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.982321    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdzw4\" (UniqueName: \"kubernetes.io/projected/f2a8421b-dbce-413f-b6ea-d4c15efb2dc1-kube-api-access-sdzw4\") pod \"multus-additional-cni-plugins-mk4bk\" (UID: \"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1\") " pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:20.982718 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.982700    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79g74\" (UniqueName: \"kubernetes.io/projected/f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf-kube-api-access-79g74\") pod \"aws-ebs-csi-driver-node-n2v82\" (UID: \"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:20.983571 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.983555    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6rbt\" (UniqueName: \"kubernetes.io/projected/30f986f8-0422-4496-af7c-42790ddb5843-kube-api-access-w6rbt\") pod \"ovnkube-node-t7t9f\" (UID: \"30f986f8-0422-4496-af7c-42790ddb5843\") " pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:20.989713 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.989682    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92dvm\" (UniqueName: \"kubernetes.io/projected/b05d5380-0b82-4ade-9481-2dcd94f82354-kube-api-access-92dvm\") pod \"node-ca-qhn45\" (UID: \"b05d5380-0b82-4ade-9481-2dcd94f82354\") " pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:20.990163 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.990141    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68774\" (UniqueName: \"kubernetes.io/projected/76cf142e-ee9d-4b89-9f27-9233e493ffad-kube-api-access-68774\") pod \"iptables-alerter-cqtrv\" (UID: \"76cf142e-ee9d-4b89-9f27-9233e493ffad\") " pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:20.990587 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.990565    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7jgw\" (UniqueName: \"kubernetes.io/projected/493fe052-d0fd-4f5d-870b-757109e4dd9d-kube-api-access-d7jgw\") pod \"tuned-bbfcc\" (UID: \"493fe052-d0fd-4f5d-870b-757109e4dd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:20.990847 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.990829    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-542w4\" (UniqueName: \"kubernetes.io/projected/fa35364e-53e3-4b89-a7bf-584d76ce3005-kube-api-access-542w4\") pod \"node-resolver-n6jp9\" (UID: \"fa35364e-53e3-4b89-a7bf-584d76ce3005\") " pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:20.992949 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:20.992925    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m5w9\" (UniqueName: \"kubernetes.io/projected/b8a3215b-b208-420a-82f1-4907c79efac1-kube-api-access-4m5w9\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:21.153216 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.153129    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82"
Jul 06 22:46:21.163362 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.163333    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bbfcc"
Jul 06 22:46:21.173117 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.173084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qhn45"
Jul 06 22:46:21.179358 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.179334    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mk4bk"
Jul 06 22:46:21.187008 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.186981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pmslq"
Jul 06 22:46:21.196669 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.196631    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cqtrv"
Jul 06 22:46:21.204423 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.204397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:21.212275 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.212250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:21.218158 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.218134    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n6jp9"
Jul 06 22:46:21.481147 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.481041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:21.481147 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.481139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:21.481389 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.481181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:21.481389 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:21.481221    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:21.481389 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:21.481313    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs podName:b8a3215b-b208-420a-82f1-4907c79efac1 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:22.481287557 +0000 UTC m=+4.023618715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs") pod "network-metrics-daemon-4jvqw" (UID: "b8a3215b-b208-420a-82f1-4907c79efac1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:21.481389 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:21.481312    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:21.481389 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:21.481380    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:46:21.481598 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:21.481397    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:46:21.481598 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:21.481385    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret podName:d7e29fc5-f57a-45bd-b114-f48b56ea7b6c nodeName:}" failed. No retries permitted until 2026-07-06 22:46:22.481366272 +0000 UTC m=+4.023697437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret") pod "global-pull-secret-syncer-mgsdw" (UID: "d7e29fc5-f57a-45bd-b114-f48b56ea7b6c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:21.481598 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:21.481409    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8cqjz for pod openshift-network-diagnostics/network-check-target-72np5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:21.481598 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:21.481453    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz podName:5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b nodeName:}" failed. No retries permitted until 2026-07-06 22:46:22.481438243 +0000 UTC m=+4.023769407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cqjz" (UniqueName: "kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz") pod "network-check-target-72np5" (UID: "5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:21.900732 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.900650    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:41:20 +0000 UTC" deadline="2028-03-11 03:43:14.924763081 +0000 UTC"
Jul 06 22:46:21.900732 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.900681    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14716h56m53.024085809s"
Jul 06 22:46:21.958127 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:21.958094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:21.958335 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:21.958208    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:22.055443 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:22.055405    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf70bec04_4f5e_4a2a_8f9a_3bb5af3bdbdf.slice/crio-d8c2d95cdc226d5cc82721e62666fd8a3637fae979b907681ec0d993b69a0f2d WatchSource:0}: Error finding container d8c2d95cdc226d5cc82721e62666fd8a3637fae979b907681ec0d993b69a0f2d: Status 404 returned error can't find the container with id d8c2d95cdc226d5cc82721e62666fd8a3637fae979b907681ec0d993b69a0f2d
Jul 06 22:46:22.057443 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:22.057023    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493fe052_d0fd_4f5d_870b_757109e4dd9d.slice/crio-ac5be343882b98cb7fdf1518aec8a927b79018619efe876d5352e0287b67a9bb WatchSource:0}: Error finding container ac5be343882b98cb7fdf1518aec8a927b79018619efe876d5352e0287b67a9bb: Status 404 returned error can't find the container with id ac5be343882b98cb7fdf1518aec8a927b79018619efe876d5352e0287b67a9bb
Jul 06 22:46:22.060333 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:22.060312    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5403f15_1e81_4b5d_b409_6b18e29e201f.slice/crio-639d438ca7fd024c657cbedbaeed9a621478c04d2805ed8595dc0fd8d0a93cc7 WatchSource:0}: Error finding container 639d438ca7fd024c657cbedbaeed9a621478c04d2805ed8595dc0fd8d0a93cc7: Status 404 returned error can't find the container with id 639d438ca7fd024c657cbedbaeed9a621478c04d2805ed8595dc0fd8d0a93cc7
Jul 06 22:46:22.061151 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:22.061129    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76cf142e_ee9d_4b89_9f27_9233e493ffad.slice/crio-9048fd6b47aefb5dc60f5a114c94e5e8ca73e32a252b78c178285280b293856f WatchSource:0}: Error finding container 9048fd6b47aefb5dc60f5a114c94e5e8ca73e32a252b78c178285280b293856f: Status 404 returned error can't find the container with id 9048fd6b47aefb5dc60f5a114c94e5e8ca73e32a252b78c178285280b293856f
Jul 06 22:46:22.489088 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:22.488850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:22.489248 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.489102    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:22.489248 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:22.489118    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:22.489248 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:22.489155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:22.489248 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.489185    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs podName:b8a3215b-b208-420a-82f1-4907c79efac1 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:24.489162323 +0000 UTC m=+6.031493474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs") pod "network-metrics-daemon-4jvqw" (UID: "b8a3215b-b208-420a-82f1-4907c79efac1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:22.489477 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.489260    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:46:22.489477 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.489285    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:46:22.489477 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.489296    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8cqjz for pod openshift-network-diagnostics/network-check-target-72np5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:22.489477 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.489343    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz podName:5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b nodeName:}" failed. No retries permitted until 2026-07-06 22:46:24.48932891 +0000 UTC m=+6.031660074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cqjz" (UniqueName: "kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz") pod "network-check-target-72np5" (UID: "5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:22.489477 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.489464    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:22.489757 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.489502    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret podName:d7e29fc5-f57a-45bd-b114-f48b56ea7b6c nodeName:}" failed. No retries permitted until 2026-07-06 22:46:24.48949028 +0000 UTC m=+6.031821428 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret") pod "global-pull-secret-syncer-mgsdw" (UID: "d7e29fc5-f57a-45bd-b114-f48b56ea7b6c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:22.901301 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:22.901107    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:41:20 +0000 UTC" deadline="2028-01-16 03:41:28.679551053 +0000 UTC"
Jul 06 22:46:22.901301 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:22.901170    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13396h55m5.77838558s"
Jul 06 22:46:22.961461 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:22.960800    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:22.961461 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.960941    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:22.961461 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:22.961025    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:22.961461 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:22.961098    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:22.980751 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:22.980711    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cqtrv" event={"ID":"76cf142e-ee9d-4b89-9f27-9233e493ffad","Type":"ContainerStarted","Data":"9048fd6b47aefb5dc60f5a114c94e5e8ca73e32a252b78c178285280b293856f"}
Jul 06 22:46:22.991738 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:22.991698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7nkqp" event={"ID":"b5403f15-1e81-4b5d-b409-6b18e29e201f","Type":"ContainerStarted","Data":"639d438ca7fd024c657cbedbaeed9a621478c04d2805ed8595dc0fd8d0a93cc7"}
Jul 06 22:46:23.000923 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.000885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bbfcc" event={"ID":"493fe052-d0fd-4f5d-870b-757109e4dd9d","Type":"ContainerStarted","Data":"ac5be343882b98cb7fdf1518aec8a927b79018619efe876d5352e0287b67a9bb"}
Jul 06 22:46:23.008243 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.008081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n6jp9" event={"ID":"fa35364e-53e3-4b89-a7bf-584d76ce3005","Type":"ContainerStarted","Data":"6b64783398b37977bdbba1a97eafc6bc01033548284767997cf011b0024e3c8b"}
Jul 06 22:46:23.018494 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.018443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mk4bk" event={"ID":"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1","Type":"ContainerStarted","Data":"1168495a1358559cf28c85e49fafaa21a7d9239ec87d13f403364f37696fd4e8"}
Jul 06 22:46:23.036238 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.036201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82" event={"ID":"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf","Type":"ContainerStarted","Data":"d8c2d95cdc226d5cc82721e62666fd8a3637fae979b907681ec0d993b69a0f2d"}
Jul 06 22:46:23.044211 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.044174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal" event={"ID":"3383faa9d76785b568517c84a7fb072b","Type":"ContainerStarted","Data":"a728a05c87bc1e179eadea77bbe52a872c8ddbea4c6c39a592fe8497ab9ce992"}
Jul 06 22:46:23.051328 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.050696    2563 generic.go:356] "Generic (PLEG): container finished" podID="058ae59d5825d565944e6642e3273d0e" containerID="b309a4e65fb8e638188833596f4e937b8504df5e4bb3640232490982177d9809" exitCode=0
Jul 06 22:46:23.051328 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.050925    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal" event={"ID":"058ae59d5825d565944e6642e3273d0e","Type":"ContainerDied","Data":"b309a4e65fb8e638188833596f4e937b8504df5e4bb3640232490982177d9809"}
Jul 06 22:46:23.060745 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.058890    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" event={"ID":"30f986f8-0422-4496-af7c-42790ddb5843","Type":"ContainerStarted","Data":"be7485088233e860779763652b7f49fe85974eb6cfe746aa697fa35ceaa48620"}
Jul 06 22:46:23.075772 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.075715    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-53.ec2.internal" podStartSLOduration=3.07569459 podStartE2EDuration="3.07569459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:23.060554477 +0000 UTC m=+4.602885648" watchObservedRunningTime="2026-07-06 22:46:23.07569459 +0000 UTC m=+4.618025761"
Jul 06 22:46:23.076784 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.076753    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pmslq" event={"ID":"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2","Type":"ContainerStarted","Data":"6d09e2a6561f7b93791590f4e2afe0f8a5950376223fde09e49029392865ffcf"}
Jul 06 22:46:23.091845 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.091803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qhn45" event={"ID":"b05d5380-0b82-4ade-9481-2dcd94f82354","Type":"ContainerStarted","Data":"8c5cd587996404c2636061ab212ebb339c2dcd05b1f9aacd69ef1dfe534c24b0"}
Jul 06 22:46:23.957629 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:23.957593    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:23.958144 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:23.957748    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:24.102787 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:24.102699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal" event={"ID":"058ae59d5825d565944e6642e3273d0e","Type":"ContainerStarted","Data":"9a34ddf0df4d8a73edaf0d2885cce333de8b7d66186a453c405ce2b814d20bab"}
Jul 06 22:46:24.507366 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:24.507329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:24.507540 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:24.507394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:24.507540 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:24.507435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:24.507739 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.507585    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:46:24.507739 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.507607    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:46:24.507739 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.507621    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8cqjz for pod openshift-network-diagnostics/network-check-target-72np5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:24.507739 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.507728    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:24.507944 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.507778    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs podName:b8a3215b-b208-420a-82f1-4907c79efac1 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:28.507764512 +0000 UTC m=+10.050095664 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs") pod "network-metrics-daemon-4jvqw" (UID: "b8a3215b-b208-420a-82f1-4907c79efac1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:24.508121 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.508102    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:24.508188 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.508156    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret podName:d7e29fc5-f57a-45bd-b114-f48b56ea7b6c nodeName:}" failed. No retries permitted until 2026-07-06 22:46:28.508140622 +0000 UTC m=+10.050471775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret") pod "global-pull-secret-syncer-mgsdw" (UID: "d7e29fc5-f57a-45bd-b114-f48b56ea7b6c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:24.508489 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.508344    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz podName:5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b nodeName:}" failed. No retries permitted until 2026-07-06 22:46:28.508331832 +0000 UTC m=+10.050662980 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cqjz" (UniqueName: "kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz") pod "network-check-target-72np5" (UID: "5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:24.959653 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:24.959557    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:24.960090 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.959696    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:24.960090 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:24.960064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:24.960195 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:24.960167    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:25.957296 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:25.957244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:25.957490 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:25.957396    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:26.959429 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:26.959395    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:26.959846 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:26.959543    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:26.959846 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:26.959629    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:26.959846 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:26.959721    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:27.957482 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:27.957404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:27.957677 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:27.957535    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:28.544535 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:28.544484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:28.544952 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:28.544678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:28.544952 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:28.544743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:28.544952 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.544870    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:28.544952 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.544923    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs podName:b8a3215b-b208-420a-82f1-4907c79efac1 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:36.544905233 +0000 UTC m=+18.087236384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs") pod "network-metrics-daemon-4jvqw" (UID: "b8a3215b-b208-420a-82f1-4907c79efac1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:28.545358 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.545339    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:28.545458 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.545383    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret podName:d7e29fc5-f57a-45bd-b114-f48b56ea7b6c nodeName:}" failed. No retries permitted until 2026-07-06 22:46:36.545371414 +0000 UTC m=+18.087702562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret") pod "global-pull-secret-syncer-mgsdw" (UID: "d7e29fc5-f57a-45bd-b114-f48b56ea7b6c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:28.545458 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.545450    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:46:28.545531 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.545461    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:46:28.545531 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.545469    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8cqjz for pod openshift-network-diagnostics/network-check-target-72np5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:28.545531 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.545492    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz podName:5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b nodeName:}" failed. No retries permitted until 2026-07-06 22:46:36.54548526 +0000 UTC m=+18.087816421 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cqjz" (UniqueName: "kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz") pod "network-check-target-72np5" (UID: "5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:28.960937 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:28.960850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:28.961086 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.960993    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:28.961415 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:28.961387    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:28.961555 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:28.961499    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:29.957344 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:29.957219    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:29.966071 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:29.966030    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:30.957180 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:30.957144    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:30.957350 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:30.957292    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:30.957694 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:30.957673    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:30.958027 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:30.957783    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:31.957503 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:31.957462    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:31.957708 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:31.957601    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:32.957265 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:32.957222    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:32.957433 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:32.957376    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:32.957514 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:32.957426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:32.957561 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:32.957531    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:33.957352 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:33.957305    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:33.957771 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:33.957445    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:34.957316 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:34.957279    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:34.957481 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:34.957401    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:34.957481 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:34.957430    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:34.957820 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:34.957491    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:35.957833 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:35.957798    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:35.958313 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:35.957926    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:36.618020 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:36.617985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:36.618032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:36.618074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.618166    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.618175    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.618200    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.618220    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.618234    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8cqjz for pod openshift-network-diagnostics/network-check-target-72np5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.618249    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret podName:d7e29fc5-f57a-45bd-b114-f48b56ea7b6c nodeName:}" failed. No retries permitted until 2026-07-06 22:46:52.618227704 +0000 UTC m=+34.160558872 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret") pod "global-pull-secret-syncer-mgsdw" (UID: "d7e29fc5-f57a-45bd-b114-f48b56ea7b6c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.618271    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs podName:b8a3215b-b208-420a-82f1-4907c79efac1 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:52.618259964 +0000 UTC m=+34.160591119 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs") pod "network-metrics-daemon-4jvqw" (UID: "b8a3215b-b208-420a-82f1-4907c79efac1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:36.618297 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.618288    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz podName:5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b nodeName:}" failed. No retries permitted until 2026-07-06 22:46:52.618278332 +0000 UTC m=+34.160609484 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cqjz" (UniqueName: "kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz") pod "network-check-target-72np5" (UID: "5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:36.957977 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:36.957894    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:36.958365 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:36.957893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:36.958365 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.958042    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:36.958365 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:36.958094    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:37.957835 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:37.957801    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:37.958022 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:37.957927    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:38.957988 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:38.957951    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:38.958189 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:38.958078    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:38.958189 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:38.958096    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:38.958189 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:38.958174    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:39.957147 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:39.957108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:39.957343 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:39.957236    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:40.957244 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:40.957208    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:40.957678 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:40.957362    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:40.957678 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:40.957397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:40.957678 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:40.957460    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:41.958058 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:41.957751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:41.958698 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:41.958137    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:42.142322 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.142283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" event={"ID":"30f986f8-0422-4496-af7c-42790ddb5843","Type":"ContainerStarted","Data":"e0628a4d41c0069b707e1ec3e7bdb1067e2e66f6f6d784f11c6538397aa8e226"}
Jul 06 22:46:42.142503 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.142383    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" event={"ID":"30f986f8-0422-4496-af7c-42790ddb5843","Type":"ContainerStarted","Data":"4eb722b714f51a6fca626ef4ca56c422f4815fd16a965fe97b443a83f9b4dcce"}
Jul 06 22:46:42.142503 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.142435    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" event={"ID":"30f986f8-0422-4496-af7c-42790ddb5843","Type":"ContainerStarted","Data":"ccafc4f0146548b02136cb2d78facfa9033ce32e8c28eb2b1d7531b9b1adc52a"}
Jul 06 22:46:42.142503 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.142475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" event={"ID":"30f986f8-0422-4496-af7c-42790ddb5843","Type":"ContainerStarted","Data":"afe7f976cf038451a019801ebdb63a80a5fbd135cc4b59d39f4a80bd443e9036"}
Jul 06 22:46:42.142629 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.142519    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" event={"ID":"30f986f8-0422-4496-af7c-42790ddb5843","Type":"ContainerStarted","Data":"01f2ee601a29c482524cc91f1405e31a6deee5c9237f98bba1f3d11f7b05aafa"}
Jul 06 22:46:42.142629 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.142552    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" event={"ID":"30f986f8-0422-4496-af7c-42790ddb5843","Type":"ContainerStarted","Data":"b85ffa482ae559b2eec2da0a7d28ea1b6b238de36377796cb2b45e402e804208"}
Jul 06 22:46:42.144318 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.144286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pmslq" event={"ID":"bba4f2f3-ca52-4cb3-a21d-830bfc6815c2","Type":"ContainerStarted","Data":"c78dba965f2c3514083166dfe7a84342cadb1b487f4cb16d1a5ed974083cbd30"}
Jul 06 22:46:42.146302 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.145943    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qhn45" event={"ID":"b05d5380-0b82-4ade-9481-2dcd94f82354","Type":"ContainerStarted","Data":"ba19bf9e92318f8d4deb0417c6d7ef12a24707ac84c956bbfafc508703f07096"}
Jul 06 22:46:42.147741 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.147708    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7nkqp" event={"ID":"b5403f15-1e81-4b5d-b409-6b18e29e201f","Type":"ContainerStarted","Data":"f0b5c1a6b0a974bf1339be53ecaaae5d7bbc9b9482b02f2e4cdad07983be3ff7"}
Jul 06 22:46:42.149619 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.149597    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bbfcc" event={"ID":"493fe052-d0fd-4f5d-870b-757109e4dd9d","Type":"ContainerStarted","Data":"56be56efcff6066c57bf61a7ea73967107ead5350170c67c7ca92b542c8bd81e"}
Jul 06 22:46:42.151101 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.151080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n6jp9" event={"ID":"fa35364e-53e3-4b89-a7bf-584d76ce3005","Type":"ContainerStarted","Data":"637fe4dfa3523b5ed335686ef54916c4452bef8fb17fdf89e73f50a45f6db14c"}
Jul 06 22:46:42.152859 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.152834    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2a8421b-dbce-413f-b6ea-d4c15efb2dc1" containerID="49d2c7f8088f58d366e5ec531407671ada06e7a79aea0e3b4abfe38db8ae1277" exitCode=0
Jul 06 22:46:42.152960 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.152922    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mk4bk" event={"ID":"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1","Type":"ContainerDied","Data":"49d2c7f8088f58d366e5ec531407671ada06e7a79aea0e3b4abfe38db8ae1277"}
Jul 06 22:46:42.154728 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.154704    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82" event={"ID":"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf","Type":"ContainerStarted","Data":"62e55ecb2b2353000e495269607b73f1acefedceb0ad8231ccfde7486d65e94e"}
Jul 06 22:46:42.164070 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.164032    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-53.ec2.internal" podStartSLOduration=22.164019952 podStartE2EDuration="22.164019952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:24.118958768 +0000 UTC m=+5.661289939" watchObservedRunningTime="2026-07-06 22:46:42.164019952 +0000 UTC m=+23.706351122"
Jul 06 22:46:42.164450 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.164422    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pmslq" podStartSLOduration=3.89681094 podStartE2EDuration="23.164413166s" totalImagesPullingTime="19.267602226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:42.163785791 +0000 UTC m=+23.706116962" watchObservedRunningTime="2026-07-06 22:46:42.164413166 +0000 UTC m=+23.706744336"
Jul 06 22:46:42.193498 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.193440    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qhn45" podStartSLOduration=3.990315766 podStartE2EDuration="23.193403126s" totalImagesPullingTime="19.20308736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:42.179874066 +0000 UTC m=+23.722205236" watchObservedRunningTime="2026-07-06 22:46:42.193403126 +0000 UTC m=+23.735734300"
Jul 06 22:46:42.193882 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.193850    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7nkqp" podStartSLOduration=3.9926948060000003 podStartE2EDuration="23.193843118s" totalImagesPullingTime="19.201148312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:42.192918372 +0000 UTC m=+23.735249542" watchObservedRunningTime="2026-07-06 22:46:42.193843118 +0000 UTC m=+23.736174291"
Jul 06 22:46:42.211222 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.211160    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bbfcc" podStartSLOduration=4.004544978 podStartE2EDuration="23.211144474s" totalImagesPullingTime="19.206599496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:42.210852888 +0000 UTC m=+23.753184079" watchObservedRunningTime="2026-07-06 22:46:42.211144474 +0000 UTC m=+23.753475644"
Jul 06 22:46:42.259449 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.259388    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n6jp9" podStartSLOduration=4.053628973 podStartE2EDuration="23.25937212s" totalImagesPullingTime="19.205743147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:42.259295954 +0000 UTC m=+23.801627124" watchObservedRunningTime="2026-07-06 22:46:42.25937212 +0000 UTC m=+23.801703289"
Jul 06 22:46:42.957510 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.957477    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:42.957707 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:42.957612    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:42.957707 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.957662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:42.958937 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:42.957765    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:42.987283 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:42.987242    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 22:46:43.158993 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:43.158914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82" event={"ID":"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf","Type":"ContainerStarted","Data":"0cb3db3c65dcd20f719c6959cb03c8ce1c6372c9dfbeca25ddfad132f9ceeeb6"}
Jul 06 22:46:43.161008 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:43.160946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cqtrv" event={"ID":"76cf142e-ee9d-4b89-9f27-9233e493ffad","Type":"ContainerStarted","Data":"d63113c557fffca9b461a04dbd5d56ef940ddb180c2008775b9769bac9c494eb"}
Jul 06 22:46:43.164802 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:43.164778    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:43.165419 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:43.165402    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:43.175833 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:43.175780    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cqtrv" podStartSLOduration=4.970150637 podStartE2EDuration="24.175768215s" totalImagesPullingTime="19.205617578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:43.175617107 +0000 UTC m=+24.717948278" watchObservedRunningTime="2026-07-06 22:46:43.175768215 +0000 UTC m=+24.718099397"
Jul 06 22:46:43.936016 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:43.935919    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T22:46:42.987261904Z","UUID":"513f087f-49b2-4d6f-9336-e62ecc5a8e5f","Handler":null,"Name":"","Endpoint":""}
Jul 06 22:46:43.937571 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:43.937514    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 22:46:43.937735 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:43.937584    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 22:46:43.957658 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:43.957613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:43.957811 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:43.957761    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:44.176576 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:44.176510    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:44.177007 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:44.176983    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7nkqp"
Jul 06 22:46:44.957948 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:44.957775    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:44.958133 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:44.957775    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:44.958133 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:44.958036    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:44.958255 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:44.958137    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:45.169657 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:45.169599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" event={"ID":"30f986f8-0422-4496-af7c-42790ddb5843","Type":"ContainerStarted","Data":"00009fbfe276bfb9e0e9c879f827cdf5be5dd3999b7613b6622b4f67a534afc6"}
Jul 06 22:46:45.172285 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:45.172253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82" event={"ID":"f70bec04-4f5e-4a2a-8f9a-3bb5af3bdbdf","Type":"ContainerStarted","Data":"005b790534d188deca6f36c18a7e7b1a91b8245be2f74176edd0071d9788ac01"}
Jul 06 22:46:45.190008 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:45.189943    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n2v82" podStartSLOduration=4.506269467 podStartE2EDuration="26.189923659s" totalImagesPullingTime="21.683654192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:45.189083741 +0000 UTC m=+26.731414914" watchObservedRunningTime="2026-07-06 22:46:45.189923659 +0000 UTC m=+26.732254829"
Jul 06 22:46:45.958094 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:45.958062    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:45.958271 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:45.958185    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:46.957322 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:46.957135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:46.958190 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:46.957135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:46.958190 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:46.957431    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:46.958190 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:46.957459    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:47.180365 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:47.180327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" event={"ID":"30f986f8-0422-4496-af7c-42790ddb5843","Type":"ContainerStarted","Data":"4c7bf8e9466e6396018b79b39d545d79113aae0d4a1ec9ac8d348b91a5bc508b"}
Jul 06 22:46:47.180660 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:47.180621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:47.182248 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:47.182225    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2a8421b-dbce-413f-b6ea-d4c15efb2dc1" containerID="e4c8e95a400d71a281a5b2276c7e654fc8feb9f48bb9b9fadb83485ba6e85da6" exitCode=0
Jul 06 22:46:47.182342 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:47.182255    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mk4bk" event={"ID":"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1","Type":"ContainerDied","Data":"e4c8e95a400d71a281a5b2276c7e654fc8feb9f48bb9b9fadb83485ba6e85da6"}
Jul 06 22:46:47.196989 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:47.196963    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:47.205615 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:47.205578    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f" podStartSLOduration=8.90947106 podStartE2EDuration="28.205563102s" totalImagesPullingTime="19.296092042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:47.2043808 +0000 UTC m=+28.746711970" watchObservedRunningTime="2026-07-06 22:46:47.205563102 +0000 UTC m=+28.747894272"
Jul 06 22:46:47.957903 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:47.957867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:47.958381 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:47.957991    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:48.184874 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:48.184845    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:48.185027 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:48.184940    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:48.200673 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:48.200626    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:46:48.544949 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:48.544871    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-72np5"]
Jul 06 22:46:48.545148 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:48.544985    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:48.545148 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:48.545062    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:48.548433 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:48.548403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4jvqw"]
Jul 06 22:46:48.548555 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:48.548528    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:48.548666 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:48.548621    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:48.548961 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:48.548942    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mgsdw"]
Jul 06 22:46:48.549041 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:48.549032    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:48.549143 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:48.549125    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:49.188629 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:49.188593    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2a8421b-dbce-413f-b6ea-d4c15efb2dc1" containerID="e499a94f67ef2e05f8855724c70de3565910726705d9b8e00f6edbf057f18120" exitCode=0
Jul 06 22:46:49.189028 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:49.188679    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mk4bk" event={"ID":"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1","Type":"ContainerDied","Data":"e499a94f67ef2e05f8855724c70de3565910726705d9b8e00f6edbf057f18120"}
Jul 06 22:46:49.958458 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:49.957972    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:49.958458 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:49.957998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:49.958458 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:49.957998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:49.958458 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:49.958102    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:49.958458 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:49.958187    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:49.958458 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:49.958280    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:51.197654 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:51.197450    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2a8421b-dbce-413f-b6ea-d4c15efb2dc1" containerID="95e3c5afea65f7adb46aca85a3bd684626f31bb55358de501defbe3cc5356798" exitCode=0
Jul 06 22:46:51.198130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:51.197536    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mk4bk" event={"ID":"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1","Type":"ContainerDied","Data":"95e3c5afea65f7adb46aca85a3bd684626f31bb55358de501defbe3cc5356798"}
Jul 06 22:46:51.957699 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:51.957661    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:51.957699 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:51.957691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:51.957914 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:51.957784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:51.957914 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:51.957805    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:51.957914 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:51.957866    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:51.958036 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:51.957939    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:52.632863 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:52.632823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:52.632884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:52.632938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:52.632996    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:52.633065    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:52.633079    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:52.633098    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:52.633111    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8cqjz for pod openshift-network-diagnostics/network-check-target-72np5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:52.633073    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs podName:b8a3215b-b208-420a-82f1-4907c79efac1 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:24.633050887 +0000 UTC m=+66.175382051 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs") pod "network-metrics-daemon-4jvqw" (UID: "b8a3215b-b208-420a-82f1-4907c79efac1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:52.633154    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret podName:d7e29fc5-f57a-45bd-b114-f48b56ea7b6c nodeName:}" failed. No retries permitted until 2026-07-06 22:47:24.633136558 +0000 UTC m=+66.175467709 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret") pod "global-pull-secret-syncer-mgsdw" (UID: "d7e29fc5-f57a-45bd-b114-f48b56ea7b6c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:46:52.633400 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:52.633169    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz podName:5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b nodeName:}" failed. No retries permitted until 2026-07-06 22:47:24.633160933 +0000 UTC m=+66.175492087 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cqjz" (UniqueName: "kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz") pod "network-check-target-72np5" (UID: "5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:46:53.957655 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:53.957599    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:53.958032 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:53.957734    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgsdw" podUID="d7e29fc5-f57a-45bd-b114-f48b56ea7b6c"
Jul 06 22:46:53.958032 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:53.957895    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:53.958032 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:53.957950    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jvqw" podUID="b8a3215b-b208-420a-82f1-4907c79efac1"
Jul 06 22:46:53.958266 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:53.958245    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:53.958386 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:53.958326    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72np5" podUID="5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b"
Jul 06 22:46:55.284578 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.284552    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-53.ec2.internal" event="NodeReady"
Jul 06 22:46:55.285104 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.284698    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 22:46:55.330825 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.330802    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-548676b5f5-8hx7m"]
Jul 06 22:46:55.332601 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.332581    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.335435 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.335290    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5qtpz\""
Jul 06 22:46:55.335435 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.335365    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 22:46:55.335554 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.335535    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 22:46:55.335791 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.335774    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 22:46:55.342619 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.342587    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2jb5z"]
Jul 06 22:46:55.343739 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.343702    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 22:46:55.345222 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.345202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2jb5z"
Jul 06 22:46:55.346929 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.346909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-548676b5f5-8hx7m"]
Jul 06 22:46:55.347588 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.347564    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 22:46:55.347963 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.347937    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 22:46:55.348039 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.347946    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4v9t\""
Jul 06 22:46:55.348039 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.347982    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 22:46:55.350408 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.350387    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4dnvj"]
Jul 06 22:46:55.352524 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.352505    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.353783 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.353766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2jb5z"]
Jul 06 22:46:55.354795 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.354779    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 22:46:55.355005 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.354992    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ctk6j\""
Jul 06 22:46:55.355082 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.355067    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 22:46:55.365969 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.365940    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4dnvj"]
Jul 06 22:46:55.453510 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.453692 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-certificates\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.453692 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d80dea7-f05c-4ea3-ad20-66f0645412af-cert\") pod \"ingress-canary-2jb5z\" (UID: \"2d80dea7-f05c-4ea3-ad20-66f0645412af\") " pod="openshift-ingress-canary/ingress-canary-2jb5z"
Jul 06 22:46:55.453692 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-installation-pull-secrets\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.453800 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-image-registry-private-configuration\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.453800 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69504cda-3d7b-4840-9b90-905a62f84cf3-ca-trust-extracted\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.453800 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tddm9\" (UniqueName: \"kubernetes.io/projected/2d80dea7-f05c-4ea3-ad20-66f0645412af-kube-api-access-tddm9\") pod \"ingress-canary-2jb5z\" (UID: \"2d80dea7-f05c-4ea3-ad20-66f0645412af\") " pod="openshift-ingress-canary/ingress-canary-2jb5z"
Jul 06 22:46:55.453800 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-config-volume\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.453800 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-trusted-ca\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.453944 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.453944 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-bound-sa-token\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.453944 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2vdb\" (UniqueName: \"kubernetes.io/projected/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-kube-api-access-q2vdb\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.453944 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz8l6\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-kube-api-access-jz8l6\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.453944 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.453932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-tmp-dir\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.479466 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.479433    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:46:55.554456 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-config-volume\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.554456 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-trusted-ca\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.554756 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.554756 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554502    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-bound-sa-token\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.554756 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2vdb\" (UniqueName: \"kubernetes.io/projected/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-kube-api-access-q2vdb\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.554756 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554541    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz8l6\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-kube-api-access-jz8l6\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.554756 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:55.554608    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:46:55.554756 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:55.554702    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls podName:b615cfe7-ddc3-4a28-8c5a-5e0fe646960a nodeName:}" failed. No retries permitted until 2026-07-06 22:46:56.054682397 +0000 UTC m=+37.597013563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls") pod "dns-default-4dnvj" (UID: "b615cfe7-ddc3-4a28-8c5a-5e0fe646960a") : secret "dns-default-metrics-tls" not found
Jul 06 22:46:55.554756 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-tmp-dir\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.555088 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.555088 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-certificates\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.555088 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d80dea7-f05c-4ea3-ad20-66f0645412af-cert\") pod \"ingress-canary-2jb5z\" (UID: \"2d80dea7-f05c-4ea3-ad20-66f0645412af\") " pod="openshift-ingress-canary/ingress-canary-2jb5z"
Jul 06 22:46:55.555088 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.554969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-installation-pull-secrets\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.555088 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.555027    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-tmp-dir\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.555088 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.555031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-image-registry-private-configuration\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.555088 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:55.554972    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:46:55.555088 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:55.555063    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548676b5f5-8hx7m: secret "image-registry-tls" not found
Jul 06 22:46:55.555088 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.555068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69504cda-3d7b-4840-9b90-905a62f84cf3-ca-trust-extracted\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.555564 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.555098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tddm9\" (UniqueName: \"kubernetes.io/projected/2d80dea7-f05c-4ea3-ad20-66f0645412af-kube-api-access-tddm9\") pod \"ingress-canary-2jb5z\" (UID: \"2d80dea7-f05c-4ea3-ad20-66f0645412af\") " pod="openshift-ingress-canary/ingress-canary-2jb5z"
Jul 06 22:46:55.555564 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:55.555139    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls podName:69504cda-3d7b-4840-9b90-905a62f84cf3 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:56.055124309 +0000 UTC m=+37.597455474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls") pod "image-registry-548676b5f5-8hx7m" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3") : secret "image-registry-tls" not found
Jul 06 22:46:55.555564 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.555166    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-config-volume\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.555564 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.555433    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-certificates\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.555564 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.555452    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69504cda-3d7b-4840-9b90-905a62f84cf3-ca-trust-extracted\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.555822 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.555625    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-trusted-ca\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.558830 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.558808    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-installation-pull-secrets\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.558830 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.558816    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d80dea7-f05c-4ea3-ad20-66f0645412af-cert\") pod \"ingress-canary-2jb5z\" (UID: \"2d80dea7-f05c-4ea3-ad20-66f0645412af\") " pod="openshift-ingress-canary/ingress-canary-2jb5z"
Jul 06 22:46:55.558962 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.558881    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-image-registry-private-configuration\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.566036 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.566015    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tddm9\" (UniqueName: \"kubernetes.io/projected/2d80dea7-f05c-4ea3-ad20-66f0645412af-kube-api-access-tddm9\") pod \"ingress-canary-2jb5z\" (UID: \"2d80dea7-f05c-4ea3-ad20-66f0645412af\") " pod="openshift-ingress-canary/ingress-canary-2jb5z"
Jul 06 22:46:55.566272 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.566244    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2vdb\" (UniqueName: \"kubernetes.io/projected/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-kube-api-access-q2vdb\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:55.566650 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.566618    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz8l6\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-kube-api-access-jz8l6\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.567560 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.567542    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-bound-sa-token\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:55.568611 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.568570    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:46:55.690024 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.689986    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2jb5z"
Jul 06 22:46:55.826238 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.826046    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2jb5z"]
Jul 06 22:46:55.830049 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:46:55.830022    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d80dea7_f05c_4ea3_ad20_66f0645412af.slice/crio-b8168f4389193b9738288d21cac1b14d28fbc3fdd398e38f827ec4482d120aba WatchSource:0}: Error finding container b8168f4389193b9738288d21cac1b14d28fbc3fdd398e38f827ec4482d120aba: Status 404 returned error can't find the container with id b8168f4389193b9738288d21cac1b14d28fbc3fdd398e38f827ec4482d120aba
Jul 06 22:46:55.957925 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.957886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:46:55.958054 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.957952    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:46:55.958054 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.957967    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:46:55.960822 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.960794    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 22:46:55.960957 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.960894    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 22:46:55.960957 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.960921    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vmht5\""
Jul 06 22:46:55.961044 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.960999    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 22:46:55.961732 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.961712    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 22:46:55.961793 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:55.961748    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j9ddx\""
Jul 06 22:46:56.058859 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:56.058770    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:56.059024 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:56.058923    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:46:56.059024 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:56.058976    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls podName:b615cfe7-ddc3-4a28-8c5a-5e0fe646960a nodeName:}" failed. No retries permitted until 2026-07-06 22:46:57.058961251 +0000 UTC m=+38.601292399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls") pod "dns-default-4dnvj" (UID: "b615cfe7-ddc3-4a28-8c5a-5e0fe646960a") : secret "dns-default-metrics-tls" not found
Jul 06 22:46:56.059024 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:56.059000    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:56.059172 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:56.059115    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:46:56.059172 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:56.059127    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548676b5f5-8hx7m: secret "image-registry-tls" not found
Jul 06 22:46:56.059259 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:56.059209    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls podName:69504cda-3d7b-4840-9b90-905a62f84cf3 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:57.059197237 +0000 UTC m=+38.601528398 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls") pod "image-registry-548676b5f5-8hx7m" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3") : secret "image-registry-tls" not found
Jul 06 22:46:56.213093 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:56.213054    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2a8421b-dbce-413f-b6ea-d4c15efb2dc1" containerID="f006bd3ec661aca2780939a15dfa0cc4a2801ee6b2e38fb51feab0f07b36efdc" exitCode=0
Jul 06 22:46:56.213293 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:56.213124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mk4bk" event={"ID":"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1","Type":"ContainerDied","Data":"f006bd3ec661aca2780939a15dfa0cc4a2801ee6b2e38fb51feab0f07b36efdc"}
Jul 06 22:46:56.214246 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:56.214226    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2jb5z" event={"ID":"2d80dea7-f05c-4ea3-ad20-66f0645412af","Type":"ContainerStarted","Data":"b8168f4389193b9738288d21cac1b14d28fbc3fdd398e38f827ec4482d120aba"}
Jul 06 22:46:57.068099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:57.068066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:57.068528 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:57.068199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:57.068528 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:57.068232    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:46:57.068528 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:57.068325    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:46:57.068528 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:57.068336    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548676b5f5-8hx7m: secret "image-registry-tls" not found
Jul 06 22:46:57.068528 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:57.068382    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls podName:69504cda-3d7b-4840-9b90-905a62f84cf3 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:59.068367601 +0000 UTC m=+40.610698762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls") pod "image-registry-548676b5f5-8hx7m" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3") : secret "image-registry-tls" not found
Jul 06 22:46:57.068528 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:57.068466    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls podName:b615cfe7-ddc3-4a28-8c5a-5e0fe646960a nodeName:}" failed. No retries permitted until 2026-07-06 22:46:59.068425091 +0000 UTC m=+40.610756279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls") pod "dns-default-4dnvj" (UID: "b615cfe7-ddc3-4a28-8c5a-5e0fe646960a") : secret "dns-default-metrics-tls" not found
Jul 06 22:46:57.220847 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:57.220813    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2a8421b-dbce-413f-b6ea-d4c15efb2dc1" containerID="8ca21def3452ad6919694a12dfcc65b06b16a85f4498b80dfd6a507fb5715a94" exitCode=0
Jul 06 22:46:57.220847 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:57.220859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mk4bk" event={"ID":"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1","Type":"ContainerDied","Data":"8ca21def3452ad6919694a12dfcc65b06b16a85f4498b80dfd6a507fb5715a94"}
Jul 06 22:46:58.226833 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:58.226622    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mk4bk" event={"ID":"f2a8421b-dbce-413f-b6ea-d4c15efb2dc1","Type":"ContainerStarted","Data":"a59f09514ff1b0d1c98e7e4943b03d8320d3e70e3848d8e6ce01426cd9a9cb24"}
Jul 06 22:46:58.248442 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:58.248396    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mk4bk" podStartSLOduration=9.428531041 podStartE2EDuration="39.2483699s" totalImagesPullingTime="29.819838859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:58.24671702 +0000 UTC m=+39.789048191" watchObservedRunningTime="2026-07-06 22:46:58.2483699 +0000 UTC m=+39.790701069"
Jul 06 22:46:58.368013 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:58.367931    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:46:58.425094 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:58.425059    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:46:58.655283 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:58.655194    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:46:58.756252 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:58.756202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:46:59.087123 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:59.087081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:46:59.087280 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:59.087142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:46:59.087280 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:59.087240    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:46:59.087280 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:59.087248    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:46:59.087280 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:59.087265    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548676b5f5-8hx7m: secret "image-registry-tls" not found
Jul 06 22:46:59.087425 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:59.087317    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls podName:b615cfe7-ddc3-4a28-8c5a-5e0fe646960a nodeName:}" failed. No retries permitted until 2026-07-06 22:47:03.087300163 +0000 UTC m=+44.629631331 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls") pod "dns-default-4dnvj" (UID: "b615cfe7-ddc3-4a28-8c5a-5e0fe646960a") : secret "dns-default-metrics-tls" not found
Jul 06 22:46:59.087425 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:46:59.087333    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls podName:69504cda-3d7b-4840-9b90-905a62f84cf3 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:03.087326861 +0000 UTC m=+44.629658009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls") pod "image-registry-548676b5f5-8hx7m" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3") : secret "image-registry-tls" not found
Jul 06 22:46:59.230111 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:59.230071    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2jb5z" event={"ID":"2d80dea7-f05c-4ea3-ad20-66f0645412af","Type":"ContainerStarted","Data":"716847ac414d2d34897041ec2ff1c0ca9aed8db200710ddf8c4effd566a9cc70"}
Jul 06 22:46:59.246062 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:46:59.246016    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2jb5z" podStartSLOduration=1.526629081 podStartE2EDuration="4.246000884s" totalImagesPullingTime="2.719371803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:46:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:59.24482945 +0000 UTC m=+40.787160611" watchObservedRunningTime="2026-07-06 22:46:59.246000884 +0000 UTC m=+40.788332053"
Jul 06 22:47:03.115584 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:03.115548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:47:03.115994 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:03.115611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:47:03.115994 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:03.115742    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:47:03.115994 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:03.115740    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:47:03.115994 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:03.115809    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls podName:b615cfe7-ddc3-4a28-8c5a-5e0fe646960a nodeName:}" failed. No retries permitted until 2026-07-06 22:47:11.115792722 +0000 UTC m=+52.658123872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls") pod "dns-default-4dnvj" (UID: "b615cfe7-ddc3-4a28-8c5a-5e0fe646960a") : secret "dns-default-metrics-tls" not found
Jul 06 22:47:03.115994 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:03.115753    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548676b5f5-8hx7m: secret "image-registry-tls" not found
Jul 06 22:47:03.115994 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:03.115871    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls podName:69504cda-3d7b-4840-9b90-905a62f84cf3 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:11.11585852 +0000 UTC m=+52.658189668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls") pod "image-registry-548676b5f5-8hx7m" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3") : secret "image-registry-tls" not found
Jul 06 22:47:03.167357 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:03.167323    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:03.227380 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:03.227343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:06.702903 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:06.702864    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:06.755802 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:06.755770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:07.317575 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:07.317532    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:07.367901 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:07.367862    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:07.986865 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:07.986835    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9"]
Jul 06 22:47:07.994984 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:07.994953    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9"
Jul 06 22:47:07.997166 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:07.997141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9"]
Jul 06 22:47:07.997548 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:07.997531    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-g8gzt\""
Jul 06 22:47:08.050092 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.050055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqr8d\" (UniqueName: \"kubernetes.io/projected/35ac541e-c33f-461e-9200-fbab544a282d-kube-api-access-xqr8d\") pod \"network-check-source-989f8b8dc-lxgs9\" (UID: \"35ac541e-c33f-461e-9200-fbab544a282d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9"
Jul 06 22:47:08.103127 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.103093    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"]
Jul 06 22:47:08.117129 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.117092    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"]
Jul 06 22:47:08.117286 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.117134    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.123379 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.120308    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 06 22:47:08.123379 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.120424    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 06 22:47:08.123379 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.120811    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7cdkn\""
Jul 06 22:47:08.123379 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.120902    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 06 22:47:08.123379 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.121876    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 06 22:47:08.126564 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.126542    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"]
Jul 06 22:47:08.126748 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.126579    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"]
Jul 06 22:47:08.126748 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.126684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.129630 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.129614    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 06 22:47:08.130770 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.130749    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 06 22:47:08.130863 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.130777    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vlpmq\""
Jul 06 22:47:08.131104 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.131086    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 06 22:47:08.132180 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.132167    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 06 22:47:08.151130 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.151100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqr8d\" (UniqueName: \"kubernetes.io/projected/35ac541e-c33f-461e-9200-fbab544a282d-kube-api-access-xqr8d\") pod \"network-check-source-989f8b8dc-lxgs9\" (UID: \"35ac541e-c33f-461e-9200-fbab544a282d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9"
Jul 06 22:47:08.151252 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.151134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cda31ca9-c242-474f-91fe-ff5065650871-config\") pod \"service-ca-operator-5d46d9ffd4-842cx\" (UID: \"cda31ca9-c242-474f-91fe-ff5065650871\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.151252 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.151159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txtjg\" (UniqueName: \"kubernetes.io/projected/cda31ca9-c242-474f-91fe-ff5065650871-kube-api-access-txtjg\") pod \"service-ca-operator-5d46d9ffd4-842cx\" (UID: \"cda31ca9-c242-474f-91fe-ff5065650871\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.151252 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.151193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc071d16-fade-4e10-9e05-b5269d503961-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-6r9bb\" (UID: \"fc071d16-fade-4e10-9e05-b5269d503961\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.151252 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.151238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc071d16-fade-4e10-9e05-b5269d503961-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-6r9bb\" (UID: \"fc071d16-fade-4e10-9e05-b5269d503961\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.151383 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.151260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cda31ca9-c242-474f-91fe-ff5065650871-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-842cx\" (UID: \"cda31ca9-c242-474f-91fe-ff5065650871\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.151383 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.151322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vps2x\" (UniqueName: \"kubernetes.io/projected/fc071d16-fade-4e10-9e05-b5269d503961-kube-api-access-vps2x\") pod \"kube-storage-version-migrator-operator-846bf668f7-6r9bb\" (UID: \"fc071d16-fade-4e10-9e05-b5269d503961\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.161709 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.161688    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqr8d\" (UniqueName: \"kubernetes.io/projected/35ac541e-c33f-461e-9200-fbab544a282d-kube-api-access-xqr8d\") pod \"network-check-source-989f8b8dc-lxgs9\" (UID: \"35ac541e-c33f-461e-9200-fbab544a282d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9"
Jul 06 22:47:08.190371 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.190332    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"]
Jul 06 22:47:08.204986 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.204955    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.206053 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.206030    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"]
Jul 06 22:47:08.209035 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.209013    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 22:47:08.209148 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.209051    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zwqrq\""
Jul 06 22:47:08.209148 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.209078    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 06 22:47:08.209148 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.208980    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 06 22:47:08.209331 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.209156    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 22:47:08.251992 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.251918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cda31ca9-c242-474f-91fe-ff5065650871-config\") pod \"service-ca-operator-5d46d9ffd4-842cx\" (UID: \"cda31ca9-c242-474f-91fe-ff5065650871\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.251992 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.251946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txtjg\" (UniqueName: \"kubernetes.io/projected/cda31ca9-c242-474f-91fe-ff5065650871-kube-api-access-txtjg\") pod \"service-ca-operator-5d46d9ffd4-842cx\" (UID: \"cda31ca9-c242-474f-91fe-ff5065650871\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.251992 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.251968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc071d16-fade-4e10-9e05-b5269d503961-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-6r9bb\" (UID: \"fc071d16-fade-4e10-9e05-b5269d503961\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.252227 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.252085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc071d16-fade-4e10-9e05-b5269d503961-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-6r9bb\" (UID: \"fc071d16-fade-4e10-9e05-b5269d503961\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.252227 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.252114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cda31ca9-c242-474f-91fe-ff5065650871-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-842cx\" (UID: \"cda31ca9-c242-474f-91fe-ff5065650871\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.252227 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.252149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.252227 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.252186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58gg9\" (UniqueName: \"kubernetes.io/projected/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-kube-api-access-58gg9\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.252427 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.252339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vps2x\" (UniqueName: \"kubernetes.io/projected/fc071d16-fade-4e10-9e05-b5269d503961-kube-api-access-vps2x\") pod \"kube-storage-version-migrator-operator-846bf668f7-6r9bb\" (UID: \"fc071d16-fade-4e10-9e05-b5269d503961\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.252427 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.252389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.252585 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.252559    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cda31ca9-c242-474f-91fe-ff5065650871-config\") pod \"service-ca-operator-5d46d9ffd4-842cx\" (UID: \"cda31ca9-c242-474f-91fe-ff5065650871\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.254472 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.254449    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cda31ca9-c242-474f-91fe-ff5065650871-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-842cx\" (UID: \"cda31ca9-c242-474f-91fe-ff5065650871\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.259758 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.259731    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc071d16-fade-4e10-9e05-b5269d503961-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-6r9bb\" (UID: \"fc071d16-fade-4e10-9e05-b5269d503961\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.259895 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.259876    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txtjg\" (UniqueName: \"kubernetes.io/projected/cda31ca9-c242-474f-91fe-ff5065650871-kube-api-access-txtjg\") pod \"service-ca-operator-5d46d9ffd4-842cx\" (UID: \"cda31ca9-c242-474f-91fe-ff5065650871\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.259957 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.259889    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc071d16-fade-4e10-9e05-b5269d503961-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-6r9bb\" (UID: \"fc071d16-fade-4e10-9e05-b5269d503961\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.261746 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.261722    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vps2x\" (UniqueName: \"kubernetes.io/projected/fc071d16-fade-4e10-9e05-b5269d503961-kube-api-access-vps2x\") pod \"kube-storage-version-migrator-operator-846bf668f7-6r9bb\" (UID: \"fc071d16-fade-4e10-9e05-b5269d503961\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.304403 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.304366    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9"
Jul 06 22:47:08.333210 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.333149    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:08.353199 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.353167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.353354 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.353247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.353354 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.353268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58gg9\" (UniqueName: \"kubernetes.io/projected/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-kube-api-access-58gg9\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.353462 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:08.353358    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:08.353462 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:08.353442    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls podName:feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:08.853421094 +0000 UTC m=+50.395752257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4czjz" (UID: "feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95") : secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:08.354095 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.354067    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.361724 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.361702    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58gg9\" (UniqueName: \"kubernetes.io/projected/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-kube-api-access-58gg9\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.432839 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.432792    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"
Jul 06 22:47:08.434937 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.434907    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:08.437668 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.437652    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"
Jul 06 22:47:08.441505 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.441484    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9"]
Jul 06 22:47:08.602373 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.602337    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb"]
Jul 06 22:47:08.610018 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:08.609992    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc071d16_fade_4e10_9e05_b5269d503961.slice/crio-52c7d9d5261d444ff2279c433d775d0246b2217bb3965c8f4cfe5ccad88c7505 WatchSource:0}: Error finding container 52c7d9d5261d444ff2279c433d775d0246b2217bb3965c8f4cfe5ccad88c7505: Status 404 returned error can't find the container with id 52c7d9d5261d444ff2279c433d775d0246b2217bb3965c8f4cfe5ccad88c7505
Jul 06 22:47:08.618929 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.618901    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx"]
Jul 06 22:47:08.622681 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:08.622631    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda31ca9_c242_474f_91fe_ff5065650871.slice/crio-745dc040d3e5ef290d89a036552edde22ecfe7a97f701f827431da0dd45b96b4 WatchSource:0}: Error finding container 745dc040d3e5ef290d89a036552edde22ecfe7a97f701f827431da0dd45b96b4: Status 404 returned error can't find the container with id 745dc040d3e5ef290d89a036552edde22ecfe7a97f701f827431da0dd45b96b4
Jul 06 22:47:08.857523 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:08.857432    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:08.857693 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:08.857588    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:08.857740 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:08.857694    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls podName:feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:09.857655811 +0000 UTC m=+51.399986971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4czjz" (UID: "feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95") : secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:09.255851 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:09.255806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx" event={"ID":"cda31ca9-c242-474f-91fe-ff5065650871","Type":"ContainerStarted","Data":"745dc040d3e5ef290d89a036552edde22ecfe7a97f701f827431da0dd45b96b4"}
Jul 06 22:47:09.257323 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:09.257287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb" event={"ID":"fc071d16-fade-4e10-9e05-b5269d503961","Type":"ContainerStarted","Data":"52c7d9d5261d444ff2279c433d775d0246b2217bb3965c8f4cfe5ccad88c7505"}
Jul 06 22:47:09.259069 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:09.259043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9" event={"ID":"35ac541e-c33f-461e-9200-fbab544a282d","Type":"ContainerStarted","Data":"4780eab64d2fab25cedd6127f8dff533b4bd5e862a6bb2149287353a0bfdeafa"}
Jul 06 22:47:09.558931 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:09.558818    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:09.615990 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:09.615869    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:09.867194 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:09.867092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:09.867364 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:09.867274    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:09.867364 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:09.867349    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls podName:feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:11.867328869 +0000 UTC m=+53.409660032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4czjz" (UID: "feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95") : secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:10.630761 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:10.630721    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:10.731366 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:10.731331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:10.839501 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:10.839462    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log" limit=1024
Jul 06 22:47:11.180109 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:11.180073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:47:11.180288 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:11.180139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:47:11.180288 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:11.180244    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:47:11.180420 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:11.180324    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls podName:b615cfe7-ddc3-4a28-8c5a-5e0fe646960a nodeName:}" failed. No retries permitted until 2026-07-06 22:47:27.180302609 +0000 UTC m=+68.722633795 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls") pod "dns-default-4dnvj" (UID: "b615cfe7-ddc3-4a28-8c5a-5e0fe646960a") : secret "dns-default-metrics-tls" not found
Jul 06 22:47:11.180420 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:11.180249    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:47:11.180420 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:11.180353    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548676b5f5-8hx7m: secret "image-registry-tls" not found
Jul 06 22:47:11.180420 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:11.180399    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls podName:69504cda-3d7b-4840-9b90-905a62f84cf3 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:27.18038733 +0000 UTC m=+68.722718499 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls") pod "image-registry-548676b5f5-8hx7m" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3") : secret "image-registry-tls" not found
Jul 06 22:47:11.885998 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:11.885958    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:11.886516 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:11.886105    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:11.886516 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:11.886186    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls podName:feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:15.886165302 +0000 UTC m=+57.428496453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4czjz" (UID: "feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95") : secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:13.270726 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.270690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb" event={"ID":"fc071d16-fade-4e10-9e05-b5269d503961","Type":"ContainerStarted","Data":"cd3aaf18d1f451abd5d8648280cb28f32f95756d104268263f214323f9ebb389"}
Jul 06 22:47:13.272325 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.272301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9" event={"ID":"35ac541e-c33f-461e-9200-fbab544a282d","Type":"ContainerStarted","Data":"886f992c5cf242bda6fd46ceea8e363e251692362880d0f66963487a661ee347"}
Jul 06 22:47:13.273988 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.273961    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx" event={"ID":"cda31ca9-c242-474f-91fe-ff5065650871","Type":"ContainerStarted","Data":"a8433d269a4437443e190420c0b0671fcccc68af30db72c986c14d4c0e1a3a7d"}
Jul 06 22:47:13.305158 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.305103    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx" podStartSLOduration=1.229469647 podStartE2EDuration="5.305083509s" totalImagesPullingTime="4.075613862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:13.304632431 +0000 UTC m=+54.846963604" watchObservedRunningTime="2026-07-06 22:47:13.305083509 +0000 UTC m=+54.847414675"
Jul 06 22:47:13.306286 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.306241    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb" podStartSLOduration=1.217889942 podStartE2EDuration="5.306227806s" totalImagesPullingTime="4.088337864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:13.290603236 +0000 UTC m=+54.832934390" watchObservedRunningTime="2026-07-06 22:47:13.306227806 +0000 UTC m=+54.848558977"
Jul 06 22:47:13.319357 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.319270    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-lxgs9" podStartSLOduration=2.056707653 podStartE2EDuration="6.319253676s" totalImagesPullingTime="4.262546023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:13.31825074 +0000 UTC m=+54.860581911" watchObservedRunningTime="2026-07-06 22:47:13.319253676 +0000 UTC m=+54.861584846"
Jul 06 22:47:13.551891 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.551807    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5bnck"]
Jul 06 22:47:13.573279 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.573241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5bnck"]
Jul 06 22:47:13.573510 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.573375    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.576085 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.576056    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 22:47:13.576284 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.576271    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 22:47:13.576532 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.576518    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xdg5g\""
Jul 06 22:47:13.576719 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.576704    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 22:47:13.577119 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.577097    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 22:47:13.601394 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.601362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-crio-socket\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.601571 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.601417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.601571 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.601445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.601571 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.601469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6wlb\" (UniqueName: \"kubernetes.io/projected/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-api-access-p6wlb\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.601758 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.601564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-data-volume\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.702091 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.702053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-data-volume\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.702289 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.702135    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-crio-socket\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.702289 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.702242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.702289 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.702254    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-crio-socket\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.702289 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.702283    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.702486 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.702318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6wlb\" (UniqueName: \"kubernetes.io/projected/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-api-access-p6wlb\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.702486 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:13.702335    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 22:47:13.702486 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:13.702406    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:14.202384856 +0000 UTC m=+55.744716007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : configmap "kube-rbac-proxy" not found
Jul 06 22:47:13.702486 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.702457    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-data-volume\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.702729 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:13.702625    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:13.702729 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:13.702700    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:14.20268173 +0000 UTC m=+55.745012892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:13.710606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.710584    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6wlb\" (UniqueName: \"kubernetes.io/projected/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-api-access-p6wlb\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:13.972629 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.972597    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz"]
Jul 06 22:47:13.994423 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.994389    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz"]
Jul 06 22:47:13.994608 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.994516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz"
Jul 06 22:47:13.996876 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.996851    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 06 22:47:13.997101 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.997087    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-r5jr8\""
Jul 06 22:47:13.997172 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:13.997150    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 06 22:47:14.105613 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:14.105578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7lqw\" (UniqueName: \"kubernetes.io/projected/87a34b5b-effd-46cd-b1d9-26d0e53630f5-kube-api-access-d7lqw\") pod \"migrator-d57d58cc9-dn2sz\" (UID: \"87a34b5b-effd-46cd-b1d9-26d0e53630f5\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz"
Jul 06 22:47:14.206438 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:14.206407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:14.206438 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:14.206440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7lqw\" (UniqueName: \"kubernetes.io/projected/87a34b5b-effd-46cd-b1d9-26d0e53630f5-kube-api-access-d7lqw\") pod \"migrator-d57d58cc9-dn2sz\" (UID: \"87a34b5b-effd-46cd-b1d9-26d0e53630f5\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz"
Jul 06 22:47:14.206688 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:14.206467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:14.206688 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:14.206543    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 22:47:14.206688 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:14.206569    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:14.206688 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:14.206612    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:15.206596539 +0000 UTC m=+56.748927686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : configmap "kube-rbac-proxy" not found
Jul 06 22:47:14.206688 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:14.206627    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:15.206621422 +0000 UTC m=+56.748952570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:14.216366 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:14.216330    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7lqw\" (UniqueName: \"kubernetes.io/projected/87a34b5b-effd-46cd-b1d9-26d0e53630f5-kube-api-access-d7lqw\") pod \"migrator-d57d58cc9-dn2sz\" (UID: \"87a34b5b-effd-46cd-b1d9-26d0e53630f5\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz"
Jul 06 22:47:14.304046 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:14.303958    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz"
Jul 06 22:47:14.436685 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:14.436610    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz"]
Jul 06 22:47:14.441705 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:14.441674    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a34b5b_effd_46cd_b1d9_26d0e53630f5.slice/crio-fd3b2081dea24c54b48fda345faa68ccf466c3ad8f180aee77e9384c74cf833f WatchSource:0}: Error finding container fd3b2081dea24c54b48fda345faa68ccf466c3ad8f180aee77e9384c74cf833f: Status 404 returned error can't find the container with id fd3b2081dea24c54b48fda345faa68ccf466c3ad8f180aee77e9384c74cf833f
Jul 06 22:47:15.214589 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:15.214543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:15.214808 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:15.214626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:15.214808 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:15.214687    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 22:47:15.214808 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:15.214779    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:17.214753379 +0000 UTC m=+58.757084550 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : configmap "kube-rbac-proxy" not found
Jul 06 22:47:15.214808 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:15.214794    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:15.215021 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:15.214852    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:17.214839377 +0000 UTC m=+58.757170581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:15.281072 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:15.281032    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz" event={"ID":"87a34b5b-effd-46cd-b1d9-26d0e53630f5","Type":"ContainerStarted","Data":"fd3b2081dea24c54b48fda345faa68ccf466c3ad8f180aee77e9384c74cf833f"}
Jul 06 22:47:15.328939 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:15.328910    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n6jp9_fa35364e-53e3-4b89-a7bf-584d76ce3005/dns-node-resolver/0.log"
Jul 06 22:47:15.921479 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:15.921447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:15.921623 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:15.921604    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:15.921702 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:15.921692    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls podName:feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:23.921677118 +0000 UTC m=+65.464008270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4czjz" (UID: "feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95") : secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:16.129224 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:16.129198    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qhn45_b05d5380-0b82-4ade-9481-2dcd94f82354/node-ca/0.log"
Jul 06 22:47:16.286877 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:16.286783    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz" event={"ID":"87a34b5b-effd-46cd-b1d9-26d0e53630f5","Type":"ContainerStarted","Data":"fa88439e8741f6f90441e33d91028ab7e596ebab56d4cfe1f5d63a3fe4981018"}
Jul 06 22:47:16.287056 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:16.286880    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz" event={"ID":"87a34b5b-effd-46cd-b1d9-26d0e53630f5","Type":"ContainerStarted","Data":"864c589fc094ecbf00965b8d8ce622ce3a9241625d37aa2251bf0468b05d728f"}
Jul 06 22:47:16.303259 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:16.303188    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-dn2sz" podStartSLOduration=2.334294292 podStartE2EDuration="3.303168046s" totalImagesPullingTime="968.873754ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:16.302885145 +0000 UTC m=+57.845216328" watchObservedRunningTime="2026-07-06 22:47:16.303168046 +0000 UTC m=+57.845499216"
Jul 06 22:47:16.729037 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:16.728985    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2jb5z_2d80dea7-f05c-4ea3-ad20-66f0645412af/serve-healthcheck-canary/0.log"
Jul 06 22:47:17.232103 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:17.232069    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:17.232103 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:17.232116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:17.232430 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:17.232212    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 22:47:17.232430 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:17.232236    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:17.232430 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:17.232280    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:21.232262674 +0000 UTC m=+62.774593825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:17.232430 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:17.232295    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:21.232287508 +0000 UTC m=+62.774618658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : configmap "kube-rbac-proxy" not found
Jul 06 22:47:20.206249 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:20.206217    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t7t9f"
Jul 06 22:47:21.267717 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:21.267675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:21.267717 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:21.267727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:21.268136 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:21.267803    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 22:47:21.268136 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:21.267821    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:21.268136 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:21.267871    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:29.267855081 +0000 UTC m=+70.810186247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : configmap "kube-rbac-proxy" not found
Jul 06 22:47:21.268136 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:21.267950    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:29.26793266 +0000 UTC m=+70.810263808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:23.994513 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:23.994471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:23.994927 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:23.994649    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:23.994927 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:23.994733    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls podName:feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:39.994716856 +0000 UTC m=+81.537048004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4czjz" (UID: "feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95") : secret "cluster-monitoring-operator-tls" not found
Jul 06 22:47:24.700528 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.700494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:47:24.700725 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.700554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:47:24.700725 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.700614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:47:24.703361 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.703321    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cqjz\" (UniqueName: \"kubernetes.io/projected/5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b-kube-api-access-8cqjz\") pod \"network-check-target-72np5\" (UID: \"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b\") " pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:47:24.703361 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.703330    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 22:47:24.703515 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.703472    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 22:47:24.711384 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:24.711363    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 22:47:24.711484 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:24.711419    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs podName:b8a3215b-b208-420a-82f1-4907c79efac1 nodeName:}" failed. No retries permitted until 2026-07-06 22:48:28.711402973 +0000 UTC m=+130.253734135 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs") pod "network-metrics-daemon-4jvqw" (UID: "b8a3215b-b208-420a-82f1-4907c79efac1") : secret "metrics-daemon-secret" not found
Jul 06 22:47:24.713436 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.713420    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d7e29fc5-f57a-45bd-b114-f48b56ea7b6c-original-pull-secret\") pod \"global-pull-secret-syncer-mgsdw\" (UID: \"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c\") " pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:47:24.768757 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.768724    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgsdw"
Jul 06 22:47:24.776760 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.776735    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j9ddx\""
Jul 06 22:47:24.784813 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.784786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:47:24.902943 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.902912    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mgsdw"]
Jul 06 22:47:24.906098 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:24.906068    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e29fc5_f57a_45bd_b114_f48b56ea7b6c.slice/crio-49d8f4bde94d9ff185e886bdfef55574a50842c1123f72099737e978245652ff WatchSource:0}: Error finding container 49d8f4bde94d9ff185e886bdfef55574a50842c1123f72099737e978245652ff: Status 404 returned error can't find the container with id 49d8f4bde94d9ff185e886bdfef55574a50842c1123f72099737e978245652ff
Jul 06 22:47:24.918303 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:24.918280    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-72np5"]
Jul 06 22:47:24.921295 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:24.921251    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b6f8655_55aa_47d2_83d2_a4dde5cc6b1b.slice/crio-b68c319715afda7893c490cbe85aeee94353e8bc8f02be37ced6796ea8c5c826 WatchSource:0}: Error finding container b68c319715afda7893c490cbe85aeee94353e8bc8f02be37ced6796ea8c5c826: Status 404 returned error can't find the container with id b68c319715afda7893c490cbe85aeee94353e8bc8f02be37ced6796ea8c5c826
Jul 06 22:47:25.318167 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:25.318063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mgsdw" event={"ID":"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c","Type":"ContainerStarted","Data":"49d8f4bde94d9ff185e886bdfef55574a50842c1123f72099737e978245652ff"}
Jul 06 22:47:25.319408 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:25.319387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-72np5" event={"ID":"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b","Type":"ContainerStarted","Data":"0efe2ad5a6216a1ab91c4e0a53f889470c647aeb0acbd0523385c8cfaef599e2"}
Jul 06 22:47:25.319516 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:25.319424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-72np5" event={"ID":"5b6f8655-55aa-47d2-83d2-a4dde5cc6b1b","Type":"ContainerStarted","Data":"b68c319715afda7893c490cbe85aeee94353e8bc8f02be37ced6796ea8c5c826"}
Jul 06 22:47:25.319553 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:25.319537    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:47:25.334508 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:25.334419    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-72np5" podStartSLOduration=66.334401596 podStartE2EDuration="1m6.334401596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:25.333767065 +0000 UTC m=+66.876098235" watchObservedRunningTime="2026-07-06 22:47:25.334401596 +0000 UTC m=+66.876732767"
Jul 06 22:47:27.221212 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:27.221174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:47:27.221582 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:27.221314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:47:27.223881 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:27.223852    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b615cfe7-ddc3-4a28-8c5a-5e0fe646960a-metrics-tls\") pod \"dns-default-4dnvj\" (UID: \"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a\") " pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:47:27.224004 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:27.223977    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"image-registry-548676b5f5-8hx7m\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") " pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:47:27.486972 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:27.486886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5qtpz\""
Jul 06 22:47:27.494962 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:27.494935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:47:27.496971 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:27.496950    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ctk6j\""
Jul 06 22:47:27.505949 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:27.505921    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:47:28.792570 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:28.792542    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-548676b5f5-8hx7m"]
Jul 06 22:47:28.796121 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:28.796094    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69504cda_3d7b_4840_9b90_905a62f84cf3.slice/crio-e966bfe70cea8de9707007e7fe72188812a5e88454c72900f9eb9e31adf2a60d WatchSource:0}: Error finding container e966bfe70cea8de9707007e7fe72188812a5e88454c72900f9eb9e31adf2a60d: Status 404 returned error can't find the container with id e966bfe70cea8de9707007e7fe72188812a5e88454c72900f9eb9e31adf2a60d
Jul 06 22:47:28.803441 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:28.803417    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4dnvj"]
Jul 06 22:47:28.805923 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:28.805896    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb615cfe7_ddc3_4a28_8c5a_5e0fe646960a.slice/crio-94e9357c15d5bb54dc659d93ace6fbdc007367943cbae09ee51595fafbdc4b1e WatchSource:0}: Error finding container 94e9357c15d5bb54dc659d93ace6fbdc007367943cbae09ee51595fafbdc4b1e: Status 404 returned error can't find the container with id 94e9357c15d5bb54dc659d93ace6fbdc007367943cbae09ee51595fafbdc4b1e
Jul 06 22:47:29.333682 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.333555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mgsdw" event={"ID":"d7e29fc5-f57a-45bd-b114-f48b56ea7b6c","Type":"ContainerStarted","Data":"c03c54d4a55a69dd0401b2011e97c7807d1fad1c2aa9c1df63a6cb7c0b30e7b0"}
Jul 06 22:47:29.335287 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.335258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m" event={"ID":"69504cda-3d7b-4840-9b90-905a62f84cf3","Type":"ContainerStarted","Data":"c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d"}
Jul 06 22:47:29.335406 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.335297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m" event={"ID":"69504cda-3d7b-4840-9b90-905a62f84cf3","Type":"ContainerStarted","Data":"e966bfe70cea8de9707007e7fe72188812a5e88454c72900f9eb9e31adf2a60d"}
Jul 06 22:47:29.335406 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.335343    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:47:29.337164 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.337126    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4dnvj" event={"ID":"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a","Type":"ContainerStarted","Data":"94e9357c15d5bb54dc659d93ace6fbdc007367943cbae09ee51595fafbdc4b1e"}
Jul 06 22:47:29.338447 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.338423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:29.338587 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.338461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:29.338674 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:29.338591    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:29.338674 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:29.338661    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls podName:4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:45.33862989 +0000 UTC m=+86.880961037 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bnck" (UID: "4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:47:29.339141 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.339116    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:29.349522 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.349469    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mgsdw" podStartSLOduration=66.237303963 podStartE2EDuration="1m10.349453631s" totalImagesPullingTime="4.112149668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:29.348447417 +0000 UTC m=+70.890778589" watchObservedRunningTime="2026-07-06 22:47:29.349453631 +0000 UTC m=+70.891784800"
Jul 06 22:47:29.369312 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:29.369250    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m" podStartSLOduration=47.369233397 podStartE2EDuration="47.369233397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:29.367826547 +0000 UTC m=+70.910157718" watchObservedRunningTime="2026-07-06 22:47:29.369233397 +0000 UTC m=+70.911564567"
Jul 06 22:47:31.345998 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:31.345960    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4dnvj" event={"ID":"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a","Type":"ContainerStarted","Data":"eafa2af6939dc805ece43fcdd87ef1b7a6ce71ed06f2a73b24cbb92e56eee096"}
Jul 06 22:47:31.346360 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:31.346023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4dnvj" event={"ID":"b615cfe7-ddc3-4a28-8c5a-5e0fe646960a","Type":"ContainerStarted","Data":"e7de228077bdab9a6533ad82435b80c3bc12dec8c59b5a1e21698d9b3bb17fe6"}
Jul 06 22:47:31.346360 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:31.346079    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:47:31.364448 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:31.364399    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4dnvj" podStartSLOduration=34.049410227 podStartE2EDuration="36.364383334s" totalImagesPullingTime="2.314973107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:46:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:31.363165114 +0000 UTC m=+72.905496285" watchObservedRunningTime="2026-07-06 22:47:31.364383334 +0000 UTC m=+72.906714546"
Jul 06 22:47:36.478096 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.478063    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"]
Jul 06 22:47:36.480721 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.480690    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.481606 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.481584    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"]
Jul 06 22:47:36.483435 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.483414    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 06 22:47:36.483547 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.483452    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 22:47:36.483624 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.483549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.484392 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.484372    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 22:47:36.484392 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.484385    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 22:47:36.486594 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.486568    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 22:47:36.488379 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.488352    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 22:47:36.488467 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.488424    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 22:47:36.488611 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.488595    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 22:47:36.500734 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.500697    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"]
Jul 06 22:47:36.504812 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.504791    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"]
Jul 06 22:47:36.597459 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.597421    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b1827ac-877a-43ee-a12f-3b813efbb20c-tmp\") pod \"klusterlet-addon-workmgr-6b54b9d8cf-6w57l\" (UID: \"5b1827ac-877a-43ee-a12f-3b813efbb20c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.597459 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.597462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5b1827ac-877a-43ee-a12f-3b813efbb20c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b54b9d8cf-6w57l\" (UID: \"5b1827ac-877a-43ee-a12f-3b813efbb20c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.597750 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.597553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.597750 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.597600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-ca\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.597750 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.597629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7629aaa5-bb4c-4505-aad4-69a1371d373a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.597750 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.597707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.597750 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.597747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42jhc\" (UniqueName: \"kubernetes.io/projected/7629aaa5-bb4c-4505-aad4-69a1371d373a-kube-api-access-42jhc\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.597943 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.597814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-hub\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.597943 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.597835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xslm2\" (UniqueName: \"kubernetes.io/projected/5b1827ac-877a-43ee-a12f-3b813efbb20c-kube-api-access-xslm2\") pod \"klusterlet-addon-workmgr-6b54b9d8cf-6w57l\" (UID: \"5b1827ac-877a-43ee-a12f-3b813efbb20c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.698777 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.698733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-hub\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.698777 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.698779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xslm2\" (UniqueName: \"kubernetes.io/projected/5b1827ac-877a-43ee-a12f-3b813efbb20c-kube-api-access-xslm2\") pod \"klusterlet-addon-workmgr-6b54b9d8cf-6w57l\" (UID: \"5b1827ac-877a-43ee-a12f-3b813efbb20c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.699030 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.698806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b1827ac-877a-43ee-a12f-3b813efbb20c-tmp\") pod \"klusterlet-addon-workmgr-6b54b9d8cf-6w57l\" (UID: \"5b1827ac-877a-43ee-a12f-3b813efbb20c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.699030 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.698834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5b1827ac-877a-43ee-a12f-3b813efbb20c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b54b9d8cf-6w57l\" (UID: \"5b1827ac-877a-43ee-a12f-3b813efbb20c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.699125 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.699044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.699125 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.699108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-ca\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.699227 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.699164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7629aaa5-bb4c-4505-aad4-69a1371d373a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.699227 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.699198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.699329 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.699233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42jhc\" (UniqueName: \"kubernetes.io/projected/7629aaa5-bb4c-4505-aad4-69a1371d373a-kube-api-access-42jhc\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.699329 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.699253    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b1827ac-877a-43ee-a12f-3b813efbb20c-tmp\") pod \"klusterlet-addon-workmgr-6b54b9d8cf-6w57l\" (UID: \"5b1827ac-877a-43ee-a12f-3b813efbb20c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.700060 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.699969    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7629aaa5-bb4c-4505-aad4-69a1371d373a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.701920 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.701889    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-ca\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.702031 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.701928    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.702031 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.701959    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.702031 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.701961    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7629aaa5-bb4c-4505-aad4-69a1371d373a-hub\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.702031 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.701985    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5b1827ac-877a-43ee-a12f-3b813efbb20c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b54b9d8cf-6w57l\" (UID: \"5b1827ac-877a-43ee-a12f-3b813efbb20c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.711072 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.711043    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xslm2\" (UniqueName: \"kubernetes.io/projected/5b1827ac-877a-43ee-a12f-3b813efbb20c-kube-api-access-xslm2\") pod \"klusterlet-addon-workmgr-6b54b9d8cf-6w57l\" (UID: \"5b1827ac-877a-43ee-a12f-3b813efbb20c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.711464 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.711443    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42jhc\" (UniqueName: \"kubernetes.io/projected/7629aaa5-bb4c-4505-aad4-69a1371d373a-kube-api-access-42jhc\") pod \"cluster-proxy-proxy-agent-7695c856f-kfnpw\" (UID: \"7629aaa5-bb4c-4505-aad4-69a1371d373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.791385 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.791285    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:36.810702 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.810670    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"
Jul 06 22:47:36.930953 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.930922    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"]
Jul 06 22:47:36.933792 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:36.933767    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b1827ac_877a_43ee_a12f_3b813efbb20c.slice/crio-73af8cb1b5afdfbae8d3bf88a291bf5bb84cd8b630e0721f3ba26ac97103e5a4 WatchSource:0}: Error finding container 73af8cb1b5afdfbae8d3bf88a291bf5bb84cd8b630e0721f3ba26ac97103e5a4: Status 404 returned error can't find the container with id 73af8cb1b5afdfbae8d3bf88a291bf5bb84cd8b630e0721f3ba26ac97103e5a4
Jul 06 22:47:36.954383 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:36.954354    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw"]
Jul 06 22:47:36.957407 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:36.957372    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7629aaa5_bb4c_4505_aad4_69a1371d373a.slice/crio-1e8cd197c76cf21ce9661a3d053962795f550d18384c4a0b4c3ed0ac8f07b38d WatchSource:0}: Error finding container 1e8cd197c76cf21ce9661a3d053962795f550d18384c4a0b4c3ed0ac8f07b38d: Status 404 returned error can't find the container with id 1e8cd197c76cf21ce9661a3d053962795f550d18384c4a0b4c3ed0ac8f07b38d
Jul 06 22:47:37.370715 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:37.370685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l" event={"ID":"5b1827ac-877a-43ee-a12f-3b813efbb20c","Type":"ContainerStarted","Data":"73af8cb1b5afdfbae8d3bf88a291bf5bb84cd8b630e0721f3ba26ac97103e5a4"}
Jul 06 22:47:37.371732 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:37.371703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw" event={"ID":"7629aaa5-bb4c-4505-aad4-69a1371d373a","Type":"ContainerStarted","Data":"1e8cd197c76cf21ce9661a3d053962795f550d18384c4a0b4c3ed0ac8f07b38d"}
Jul 06 22:47:40.029741 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:40.029596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:40.032482 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:40.032446    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4czjz\" (UID: \"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:40.317684 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:40.317592    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zwqrq\""
Jul 06 22:47:40.325008 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:40.324976    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"
Jul 06 22:47:41.352298 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:41.352250    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4dnvj"
Jul 06 22:47:42.727942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:42.727917    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz"]
Jul 06 22:47:42.730147 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:42.730117    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaa18c2_daed_4b96_8c5d_ddb3b8a4cb95.slice/crio-0bd315733fbc710960affcebc4c0be274b892e84c5fb94fe96fdcd23b9767f4f WatchSource:0}: Error finding container 0bd315733fbc710960affcebc4c0be274b892e84c5fb94fe96fdcd23b9767f4f: Status 404 returned error can't find the container with id 0bd315733fbc710960affcebc4c0be274b892e84c5fb94fe96fdcd23b9767f4f
Jul 06 22:47:43.397968 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:43.397931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw" event={"ID":"7629aaa5-bb4c-4505-aad4-69a1371d373a","Type":"ContainerStarted","Data":"a3a7aaca3743df2a287427c0ed190a14a9b84338840e6858a25502e39c0b59de"}
Jul 06 22:47:43.398172 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:43.397987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw" event={"ID":"7629aaa5-bb4c-4505-aad4-69a1371d373a","Type":"ContainerStarted","Data":"7d25b95f7cd35ae57bf804995b1fc200c6993da43c93a235d65d32dc1ca094bb"}
Jul 06 22:47:43.398172 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:43.398005    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw" event={"ID":"7629aaa5-bb4c-4505-aad4-69a1371d373a","Type":"ContainerStarted","Data":"68f049b568ee2fb01785cb55b563fe8db8b12b8c12726557e8dd0a1baf13842d"}
Jul 06 22:47:43.399857 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:43.399473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l" event={"ID":"5b1827ac-877a-43ee-a12f-3b813efbb20c","Type":"ContainerStarted","Data":"bd7855e4a01dd1ce008de232a54b2588068d99d64fed580e61e4ec6d3684f245"}
Jul 06 22:47:43.399857 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:43.399569    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:43.400654 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:43.400608    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz" event={"ID":"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95","Type":"ContainerStarted","Data":"0bd315733fbc710960affcebc4c0be274b892e84c5fb94fe96fdcd23b9767f4f"}
Jul 06 22:47:43.401672 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:43.401651    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l"
Jul 06 22:47:43.415735 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:43.415685    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7695c856f-kfnpw" podStartSLOduration=1.777432663 podStartE2EDuration="7.41566641s" totalImagesPullingTime="5.638233747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:43.413902483 +0000 UTC m=+84.956233652" watchObservedRunningTime="2026-07-06 22:47:43.41566641 +0000 UTC m=+84.957997581"
Jul 06 22:47:43.429043 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:43.428985    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b54b9d8cf-6w57l" podStartSLOduration=1.783268565 podStartE2EDuration="7.428963261s" totalImagesPullingTime="5.645694696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:43.427744889 +0000 UTC m=+84.970076062" watchObservedRunningTime="2026-07-06 22:47:43.428963261 +0000 UTC m=+84.971294431"
Jul 06 22:47:44.405331 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:44.405294    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz" event={"ID":"feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95","Type":"ContainerStarted","Data":"6d0bf5b1aab221a44be010d91dc9887a9c772adac3d4b7da4a422bace30a3a82"}
Jul 06 22:47:44.421518 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:44.421447    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4czjz" podStartSLOduration=34.859656146 podStartE2EDuration="36.421424647s" totalImagesPullingTime="1.561768501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:44.420416996 +0000 UTC m=+85.962748167" watchObservedRunningTime="2026-07-06 22:47:44.421424647 +0000 UTC m=+85.963755819"
Jul 06 22:47:45.374274 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:45.374227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:45.376750 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:45.376728    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bnck\" (UID: \"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99\") " pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:45.387259 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:45.387237    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xdg5g\""
Jul 06 22:47:45.395338 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:45.395318    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5bnck"
Jul 06 22:47:45.527837 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:45.527802    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5bnck"]
Jul 06 22:47:45.531295 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:45.531272    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2c3fa4_94c6_4be8_a2fc_ffd3f796cc99.slice/crio-0bf12872a0bd54117549a4c55cdb55dcd12b5dcffbfb2736824abcad9c2a8802 WatchSource:0}: Error finding container 0bf12872a0bd54117549a4c55cdb55dcd12b5dcffbfb2736824abcad9c2a8802: Status 404 returned error can't find the container with id 0bf12872a0bd54117549a4c55cdb55dcd12b5dcffbfb2736824abcad9c2a8802
Jul 06 22:47:46.417520 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:46.417483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5bnck" event={"ID":"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99","Type":"ContainerStarted","Data":"0bf12872a0bd54117549a4c55cdb55dcd12b5dcffbfb2736824abcad9c2a8802"}
Jul 06 22:47:47.500439 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:47.500401    2563 patch_prober.go:28] interesting pod/image-registry-548676b5f5-8hx7m container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 22:47:47.500970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:47.500474    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m" podUID="69504cda-3d7b-4840-9b90-905a62f84cf3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 22:47:49.429261 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:49.429230    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5bnck" event={"ID":"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99","Type":"ContainerStarted","Data":"644da64b491b1b2a83bb1770423d24aa3545b9c2bbb39ec64ea99026aae75f22"}
Jul 06 22:47:50.345843 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:50.345762    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:47:50.435463 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:50.435423    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5bnck" event={"ID":"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99","Type":"ContainerStarted","Data":"41074351bb01036c54c84f86902772b487c566eb34286830b029f8b85844267e"}
Jul 06 22:47:52.262681 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.261563    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z7tdr"]
Jul 06 22:47:52.279382 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.279346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.282536 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.282338    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 22:47:52.283784 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.283760    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 22:47:52.283915 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.283817    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-njcl6\""
Jul 06 22:47:52.283915 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.283855    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 22:47:52.284073 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.284055    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 22:47:52.336428 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.336392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2bxg\" (UniqueName: \"kubernetes.io/projected/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-kube-api-access-q2bxg\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.336428 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.336438    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-textfile\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.336697 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.336463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-root\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.336697 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.336535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.336697 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.336575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.336697 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.336684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-sys\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.336885 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.336720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-tls\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.336885 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.336754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-metrics-client-ca\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.336885 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.336800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-wtmp\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437165 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-root\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437239 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437300 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437300 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437259    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-root\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437300 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-sys\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437437 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-tls\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437437 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437338    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-sys\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437437 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-metrics-client-ca\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437437 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-wtmp\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437577 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:52.437436    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 22:47:52.437577 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2bxg\" (UniqueName: \"kubernetes.io/projected/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-kube-api-access-q2bxg\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437577 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:52.437488    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-tls podName:a28f9b11-6d92-42a1-a55b-28d8211f6ed2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:52.937469525 +0000 UTC m=+94.479800690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-tls") pod "node-exporter-z7tdr" (UID: "a28f9b11-6d92-42a1-a55b-28d8211f6ed2") : secret "node-exporter-tls" not found
Jul 06 22:47:52.437577 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-textfile\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.437956 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.437932    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.438104 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.438070    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-wtmp\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.438316 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.438293    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-metrics-client-ca\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.438316 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.438288    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-textfile\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.441043 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.440745    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.448690 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.448664    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2bxg\" (UniqueName: \"kubernetes.io/projected/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-kube-api-access-q2bxg\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.940742 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:52.940710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-tls\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:52.940891 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:52.940869    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 22:47:52.940972 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:47:52.940961    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-tls podName:a28f9b11-6d92-42a1-a55b-28d8211f6ed2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:53.94094499 +0000 UTC m=+95.483276161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-tls") pod "node-exporter-z7tdr" (UID: "a28f9b11-6d92-42a1-a55b-28d8211f6ed2") : secret "node-exporter-tls" not found
Jul 06 22:47:53.448330 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:53.448295    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5bnck" event={"ID":"4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99","Type":"ContainerStarted","Data":"73a9519ba5c8cf11edac543f4bb4c8e39c13e215ff2e05293065577e297932f0"}
Jul 06 22:47:53.471518 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:53.471463    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5bnck" podStartSLOduration=33.693342926 podStartE2EDuration="40.471447282s" totalImagesPullingTime="6.778104356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:47:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:53.47042742 +0000 UTC m=+95.012758590" watchObservedRunningTime="2026-07-06 22:47:53.471447282 +0000 UTC m=+95.013778502"
Jul 06 22:47:53.949631 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:53.949596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-tls\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:53.952078 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:53.952056    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a28f9b11-6d92-42a1-a55b-28d8211f6ed2-node-exporter-tls\") pod \"node-exporter-z7tdr\" (UID: \"a28f9b11-6d92-42a1-a55b-28d8211f6ed2\") " pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:54.090682 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:54.090626    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z7tdr"
Jul 06 22:47:54.101243 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:47:54.101216    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28f9b11_6d92_42a1_a55b_28d8211f6ed2.slice/crio-05774767ce852b43810c18aca672b21b8c307e98900a2409d8a0dec612653771 WatchSource:0}: Error finding container 05774767ce852b43810c18aca672b21b8c307e98900a2409d8a0dec612653771: Status 404 returned error can't find the container with id 05774767ce852b43810c18aca672b21b8c307e98900a2409d8a0dec612653771
Jul 06 22:47:54.452267 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:54.452229    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7tdr" event={"ID":"a28f9b11-6d92-42a1-a55b-28d8211f6ed2","Type":"ContainerStarted","Data":"05774767ce852b43810c18aca672b21b8c307e98900a2409d8a0dec612653771"}
Jul 06 22:47:55.457599 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:55.457568    2563 generic.go:356] "Generic (PLEG): container finished" podID="a28f9b11-6d92-42a1-a55b-28d8211f6ed2" containerID="8162641fc8337db7ce4c134dcee5696d4119be83162801bd24b7508c2fb25b93" exitCode=0
Jul 06 22:47:55.457963 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:55.457614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7tdr" event={"ID":"a28f9b11-6d92-42a1-a55b-28d8211f6ed2","Type":"ContainerDied","Data":"8162641fc8337db7ce4c134dcee5696d4119be83162801bd24b7508c2fb25b93"}
Jul 06 22:47:56.325422 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:56.325388    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-72np5"
Jul 06 22:47:56.463371 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:56.463339    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7tdr" event={"ID":"a28f9b11-6d92-42a1-a55b-28d8211f6ed2","Type":"ContainerStarted","Data":"e41a9746b37e961f679a64dadef322648bc0792a430e90c37a2082223c02be8e"}
Jul 06 22:47:56.463789 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:56.463391    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7tdr" event={"ID":"a28f9b11-6d92-42a1-a55b-28d8211f6ed2","Type":"ContainerStarted","Data":"510034ad476fabfa106e76e0703dc513ad3f4bd1ac95fc25783c47e863d9d91e"}
Jul 06 22:47:56.485942 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:47:56.485875    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z7tdr" podStartSLOduration=3.59427087 podStartE2EDuration="4.485859864s" totalImagesPullingTime="820.487354ms" totalInitContainerRuntime="71.10164ms" isStatefulPod=true podCreationTimestamp="2026-07-06 22:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:56.485150965 +0000 UTC m=+98.027482135" watchObservedRunningTime="2026-07-06 22:47:56.485859864 +0000 UTC m=+98.028191064"
Jul 06 22:48:00.785430 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:00.785387    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-548676b5f5-8hx7m"]
Jul 06 22:48:25.812153 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:25.812090    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m" podUID="69504cda-3d7b-4840-9b90-905a62f84cf3" containerName="registry" containerID="cri-o://c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d" gracePeriod=30
Jul 06 22:48:26.051810 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.051785    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:48:26.107211 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.107120    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") pod \"69504cda-3d7b-4840-9b90-905a62f84cf3\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") "
Jul 06 22:48:26.107211 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.107161    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-bound-sa-token\") pod \"69504cda-3d7b-4840-9b90-905a62f84cf3\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") "
Jul 06 22:48:26.107211 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.107205    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69504cda-3d7b-4840-9b90-905a62f84cf3-ca-trust-extracted\") pod \"69504cda-3d7b-4840-9b90-905a62f84cf3\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") "
Jul 06 22:48:26.107485 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.107233    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-image-registry-private-configuration\") pod \"69504cda-3d7b-4840-9b90-905a62f84cf3\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") "
Jul 06 22:48:26.107485 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.107276    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-trusted-ca\") pod \"69504cda-3d7b-4840-9b90-905a62f84cf3\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") "
Jul 06 22:48:26.107485 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.107304    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz8l6\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-kube-api-access-jz8l6\") pod \"69504cda-3d7b-4840-9b90-905a62f84cf3\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") "
Jul 06 22:48:26.107485 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.107361    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-certificates\") pod \"69504cda-3d7b-4840-9b90-905a62f84cf3\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") "
Jul 06 22:48:26.107485 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.107386    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-installation-pull-secrets\") pod \"69504cda-3d7b-4840-9b90-905a62f84cf3\" (UID: \"69504cda-3d7b-4840-9b90-905a62f84cf3\") "
Jul 06 22:48:26.107952 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.107914    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "69504cda-3d7b-4840-9b90-905a62f84cf3" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:48:26.108368 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.108333    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "69504cda-3d7b-4840-9b90-905a62f84cf3" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:48:26.110023 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.109976    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "69504cda-3d7b-4840-9b90-905a62f84cf3" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:48:26.110023 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.110001    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "69504cda-3d7b-4840-9b90-905a62f84cf3" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:48:26.110170 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.110073    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "69504cda-3d7b-4840-9b90-905a62f84cf3" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:48:26.110347 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.110327    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "69504cda-3d7b-4840-9b90-905a62f84cf3" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:48:26.110409 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.110355    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-kube-api-access-jz8l6" (OuterVolumeSpecName: "kube-api-access-jz8l6") pod "69504cda-3d7b-4840-9b90-905a62f84cf3" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3"). InnerVolumeSpecName "kube-api-access-jz8l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:48:26.116926 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.116901    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69504cda-3d7b-4840-9b90-905a62f84cf3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "69504cda-3d7b-4840-9b90-905a62f84cf3" (UID: "69504cda-3d7b-4840-9b90-905a62f84cf3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:48:26.208873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.208824    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69504cda-3d7b-4840-9b90-905a62f84cf3-ca-trust-extracted\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:48:26.208873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.208868    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-image-registry-private-configuration\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:48:26.208873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.208879    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-trusted-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:48:26.208873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.208889    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz8l6\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-kube-api-access-jz8l6\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:48:26.209119 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.208898    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-certificates\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:48:26.209119 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.208906    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69504cda-3d7b-4840-9b90-905a62f84cf3-installation-pull-secrets\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:48:26.209119 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.208914    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-registry-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:48:26.209119 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.208922    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69504cda-3d7b-4840-9b90-905a62f84cf3-bound-sa-token\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:48:26.562509 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.562474    2563 generic.go:356] "Generic (PLEG): container finished" podID="69504cda-3d7b-4840-9b90-905a62f84cf3" containerID="c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d" exitCode=0
Jul 06 22:48:26.562741 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.562520    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m"
Jul 06 22:48:26.562741 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.562539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m" event={"ID":"69504cda-3d7b-4840-9b90-905a62f84cf3","Type":"ContainerDied","Data":"c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d"}
Jul 06 22:48:26.562741 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.562592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548676b5f5-8hx7m" event={"ID":"69504cda-3d7b-4840-9b90-905a62f84cf3","Type":"ContainerDied","Data":"e966bfe70cea8de9707007e7fe72188812a5e88454c72900f9eb9e31adf2a60d"}
Jul 06 22:48:26.562741 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.562618    2563 scope.go:117] "RemoveContainer" containerID="c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d"
Jul 06 22:48:26.570883 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.570867    2563 scope.go:117] "RemoveContainer" containerID="c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d"
Jul 06 22:48:26.571140 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:48:26.571123    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d\": container with ID starting with c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d not found: ID does not exist" containerID="c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d"
Jul 06 22:48:26.571191 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.571148    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d"} err="failed to get container status \"c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d\": rpc error: code = NotFound desc = could not find container \"c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d\": container with ID starting with c196242e9828747fa20bb988c752414a675096214763c707540b7999f389d36d not found: ID does not exist"
Jul 06 22:48:26.582775 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.582748    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-548676b5f5-8hx7m"]
Jul 06 22:48:26.585988 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.585966    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-548676b5f5-8hx7m"]
Jul 06 22:48:26.962196 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:26.962169    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69504cda-3d7b-4840-9b90-905a62f84cf3" path="/var/lib/kubelet/pods/69504cda-3d7b-4840-9b90-905a62f84cf3/volumes"
Jul 06 22:48:28.726552 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:28.726512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:48:28.729002 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:28.728976    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a3215b-b208-420a-82f1-4907c79efac1-metrics-certs\") pod \"network-metrics-daemon-4jvqw\" (UID: \"b8a3215b-b208-420a-82f1-4907c79efac1\") " pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:48:28.981467 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:28.981372    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vmht5\""
Jul 06 22:48:28.989576 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:28.989548    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jvqw"
Jul 06 22:48:29.128204 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:29.128163    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4jvqw"]
Jul 06 22:48:29.131370 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:48:29.131344    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a3215b_b208_420a_82f1_4907c79efac1.slice/crio-c95d7e3350bb208870e429aed40e22b71e918845b066ecf37049766f8adca778 WatchSource:0}: Error finding container c95d7e3350bb208870e429aed40e22b71e918845b066ecf37049766f8adca778: Status 404 returned error can't find the container with id c95d7e3350bb208870e429aed40e22b71e918845b066ecf37049766f8adca778
Jul 06 22:48:29.577621 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:29.577580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4jvqw" event={"ID":"b8a3215b-b208-420a-82f1-4907c79efac1","Type":"ContainerStarted","Data":"c95d7e3350bb208870e429aed40e22b71e918845b066ecf37049766f8adca778"}
Jul 06 22:48:30.583421 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:30.583370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4jvqw" event={"ID":"b8a3215b-b208-420a-82f1-4907c79efac1","Type":"ContainerStarted","Data":"19b0a8d9bc0d9907992fabc9af2e5e4d2ebb02f5300de3ea817fb67de9d480d2"}
Jul 06 22:48:30.583827 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:30.583465    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4jvqw" event={"ID":"b8a3215b-b208-420a-82f1-4907c79efac1","Type":"ContainerStarted","Data":"2b9b74a4dd1fff905934cbe8911ff1803524baad8e4ee62aa067039213931c1a"}
Jul 06 22:48:30.600921 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:30.600871    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4jvqw" podStartSLOduration=130.648645472 podStartE2EDuration="2m11.60085324s" totalImagesPullingTime="952.207768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:48:30.599413389 +0000 UTC m=+132.141744561" watchObservedRunningTime="2026-07-06 22:48:30.60085324 +0000 UTC m=+132.143184410"
Jul 06 22:48:33.596797 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:33.596714    2563 generic.go:356] "Generic (PLEG): container finished" podID="fc071d16-fade-4e10-9e05-b5269d503961" containerID="cd3aaf18d1f451abd5d8648280cb28f32f95756d104268263f214323f9ebb389" exitCode=0
Jul 06 22:48:33.596797 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:33.596785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb" event={"ID":"fc071d16-fade-4e10-9e05-b5269d503961","Type":"ContainerDied","Data":"cd3aaf18d1f451abd5d8648280cb28f32f95756d104268263f214323f9ebb389"}
Jul 06 22:48:33.597276 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:33.597155    2563 scope.go:117] "RemoveContainer" containerID="cd3aaf18d1f451abd5d8648280cb28f32f95756d104268263f214323f9ebb389"
Jul 06 22:48:33.598472 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:33.598449    2563 generic.go:356] "Generic (PLEG): container finished" podID="cda31ca9-c242-474f-91fe-ff5065650871" containerID="a8433d269a4437443e190420c0b0671fcccc68af30db72c986c14d4c0e1a3a7d" exitCode=0
Jul 06 22:48:33.598603 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:33.598502    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx" event={"ID":"cda31ca9-c242-474f-91fe-ff5065650871","Type":"ContainerDied","Data":"a8433d269a4437443e190420c0b0671fcccc68af30db72c986c14d4c0e1a3a7d"}
Jul 06 22:48:33.598846 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:33.598832    2563 scope.go:117] "RemoveContainer" containerID="a8433d269a4437443e190420c0b0671fcccc68af30db72c986c14d4c0e1a3a7d"
Jul 06 22:48:34.607203 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:34.607166    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-842cx" event={"ID":"cda31ca9-c242-474f-91fe-ff5065650871","Type":"ContainerStarted","Data":"491ebbe894ce287e39c27ce8d8132f4930ecae362eb1ce5e5a57fd671f3d260d"}
Jul 06 22:48:34.608950 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:48:34.608923    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-6r9bb" event={"ID":"fc071d16-fade-4e10-9e05-b5269d503961","Type":"ContainerStarted","Data":"7f88f2cf09546b785c8bcb97a6cfcbf4b1c4040fbd6355b916957d2bb0f6857d"}
Jul 06 22:51:18.875275 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:18.875247    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 22:51:58.551558 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.551518    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"]
Jul 06 22:51:58.552113 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.551925    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69504cda-3d7b-4840-9b90-905a62f84cf3" containerName="registry"
Jul 06 22:51:58.552113 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.551942    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="69504cda-3d7b-4840-9b90-905a62f84cf3" containerName="registry"
Jul 06 22:51:58.552113 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.552015    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="69504cda-3d7b-4840-9b90-905a62f84cf3" containerName="registry"
Jul 06 22:51:58.554828 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.554808    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"
Jul 06 22:51:58.557962 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.557944    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 06 22:51:58.559003 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.558981    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fc989\""
Jul 06 22:51:58.559118 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.558984    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 06 22:51:58.568023 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.567994    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"]
Jul 06 22:51:58.623416 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.623374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58013373-86cb-4e6b-ac53-5c95588f973d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kdhbq\" (UID: \"58013373-86cb-4e6b-ac53-5c95588f973d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"
Jul 06 22:51:58.623610 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.623434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrgbs\" (UniqueName: \"kubernetes.io/projected/58013373-86cb-4e6b-ac53-5c95588f973d-kube-api-access-lrgbs\") pod \"cert-manager-cainjector-559cdd8c87-kdhbq\" (UID: \"58013373-86cb-4e6b-ac53-5c95588f973d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"
Jul 06 22:51:58.724468 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.724433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrgbs\" (UniqueName: \"kubernetes.io/projected/58013373-86cb-4e6b-ac53-5c95588f973d-kube-api-access-lrgbs\") pod \"cert-manager-cainjector-559cdd8c87-kdhbq\" (UID: \"58013373-86cb-4e6b-ac53-5c95588f973d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"
Jul 06 22:51:58.724686 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.724496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58013373-86cb-4e6b-ac53-5c95588f973d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kdhbq\" (UID: \"58013373-86cb-4e6b-ac53-5c95588f973d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"
Jul 06 22:51:58.732580 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.732558    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58013373-86cb-4e6b-ac53-5c95588f973d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kdhbq\" (UID: \"58013373-86cb-4e6b-ac53-5c95588f973d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"
Jul 06 22:51:58.733291 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.733255    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrgbs\" (UniqueName: \"kubernetes.io/projected/58013373-86cb-4e6b-ac53-5c95588f973d-kube-api-access-lrgbs\") pod \"cert-manager-cainjector-559cdd8c87-kdhbq\" (UID: \"58013373-86cb-4e6b-ac53-5c95588f973d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"
Jul 06 22:51:58.864210 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.864137    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"
Jul 06 22:51:58.996133 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:58.995957    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq"]
Jul 06 22:51:58.999116 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:51:58.999086    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58013373_86cb_4e6b_ac53_5c95588f973d.slice/crio-10d085784c94e67a0392292250c5894ad155864971c3d58bf38af518dfef4c7d WatchSource:0}: Error finding container 10d085784c94e67a0392292250c5894ad155864971c3d58bf38af518dfef4c7d: Status 404 returned error can't find the container with id 10d085784c94e67a0392292250c5894ad155864971c3d58bf38af518dfef4c7d
Jul 06 22:51:59.001037 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:59.001019    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:51:59.316123 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:51:59.316090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq" event={"ID":"58013373-86cb-4e6b-ac53-5c95588f973d","Type":"ContainerStarted","Data":"10d085784c94e67a0392292250c5894ad155864971c3d58bf38af518dfef4c7d"}
Jul 06 22:52:02.328909 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:02.328872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq" event={"ID":"58013373-86cb-4e6b-ac53-5c95588f973d","Type":"ContainerStarted","Data":"f1cf7dc7a498b436b950da218db84cd86429c4bb81bbe2d390dc95b770051087"}
Jul 06 22:52:02.346856 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:02.346805    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kdhbq" podStartSLOduration=1.4852598430000001 podStartE2EDuration="4.346791629s" totalImagesPullingTime="2.861531786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:52:02.346294421 +0000 UTC m=+343.888625591" watchObservedRunningTime="2026-07-06 22:52:02.346791629 +0000 UTC m=+343.889122800"
Jul 06 22:52:27.746386 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.746313    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"]
Jul 06 22:52:27.749167 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.749151    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.753343 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.753282    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 06 22:52:27.753343 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.753329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 06 22:52:27.753343 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.753338    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9r88k\""
Jul 06 22:52:27.753601 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.753356    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 06 22:52:27.753601 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.753338    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 06 22:52:27.753601 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.753360    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 06 22:52:27.760840 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.760817    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"]
Jul 06 22:52:27.842872 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.842841    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/30588141-2216-45d3-9dac-958c26affe46-manager-config\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.843050 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.842887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jbpm\" (UniqueName: \"kubernetes.io/projected/30588141-2216-45d3-9dac-958c26affe46-kube-api-access-9jbpm\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.843050 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.842954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/30588141-2216-45d3-9dac-958c26affe46-metrics-cert\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.843050 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.842993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30588141-2216-45d3-9dac-958c26affe46-cert\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.943917 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.943870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jbpm\" (UniqueName: \"kubernetes.io/projected/30588141-2216-45d3-9dac-958c26affe46-kube-api-access-9jbpm\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.943917 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.943928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/30588141-2216-45d3-9dac-958c26affe46-metrics-cert\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.944175 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.943954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30588141-2216-45d3-9dac-958c26affe46-cert\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.944175 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.943990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/30588141-2216-45d3-9dac-958c26affe46-manager-config\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.944580 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.944560    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/30588141-2216-45d3-9dac-958c26affe46-manager-config\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.946688 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.946669    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30588141-2216-45d3-9dac-958c26affe46-cert\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.946763 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.946732    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/30588141-2216-45d3-9dac-958c26affe46-metrics-cert\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:27.952258 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:27.952238    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jbpm\" (UniqueName: \"kubernetes.io/projected/30588141-2216-45d3-9dac-958c26affe46-kube-api-access-9jbpm\") pod \"lws-controller-manager-5747d5c666-g6gkg\" (UID: \"30588141-2216-45d3-9dac-958c26affe46\") " pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:28.059042 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:28.058935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:28.187047 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:28.187016    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"]
Jul 06 22:52:28.190424 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:52:28.190388    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30588141_2216_45d3_9dac_958c26affe46.slice/crio-0ca4e808a7e93ef21b82e658c0a2d6cdbb0ddc61c4dda44465f4d4c8c0221764 WatchSource:0}: Error finding container 0ca4e808a7e93ef21b82e658c0a2d6cdbb0ddc61c4dda44465f4d4c8c0221764: Status 404 returned error can't find the container with id 0ca4e808a7e93ef21b82e658c0a2d6cdbb0ddc61c4dda44465f4d4c8c0221764
Jul 06 22:52:28.418836 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:28.418747    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg" event={"ID":"30588141-2216-45d3-9dac-958c26affe46","Type":"ContainerStarted","Data":"0ca4e808a7e93ef21b82e658c0a2d6cdbb0ddc61c4dda44465f4d4c8c0221764"}
Jul 06 22:52:31.432380 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:31.432344    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg" event={"ID":"30588141-2216-45d3-9dac-958c26affe46","Type":"ContainerStarted","Data":"b067d29bae744770b1e905d8c376b6aeb25ef5c9d9523196cb4961ff9c2fee30"}
Jul 06 22:52:31.432827 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:31.432479    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:31.450866 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:31.450818    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg" podStartSLOduration=1.548218742 podStartE2EDuration="4.450800676s" totalImagesPullingTime="2.902581934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:52:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:52:31.449757477 +0000 UTC m=+372.992088651" watchObservedRunningTime="2026-07-06 22:52:31.450800676 +0000 UTC m=+372.993131846"
Jul 06 22:52:36.344210 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.344177    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"]
Jul 06 22:52:36.349293 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.349268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.352323 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.352299    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 22:52:36.352770 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.352746    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dx78r\""
Jul 06 22:52:36.352874 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.352792    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 22:52:36.352959 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.352940    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 22:52:36.357133 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.357116    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 22:52:36.371890 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.371863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"]
Jul 06 22:52:36.504801 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.504766    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-tmp\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.504980 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.504811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.504980 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.504838    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqrk9\" (UniqueName: \"kubernetes.io/projected/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-kube-api-access-jqrk9\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.504980 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.504884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.504980 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.504903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-manifests\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.606150 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.606051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrk9\" (UniqueName: \"kubernetes.io/projected/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-kube-api-access-jqrk9\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.606150 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.606117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.606150 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.606145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-manifests\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.606470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.606189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-tmp\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.606470 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.606224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.606597 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.606547    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-manifests\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.608732 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.608707    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-tmp\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.608829 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.608755    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.608870 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.608850    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.619099 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.619076    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrk9\" (UniqueName: \"kubernetes.io/projected/5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc-kube-api-access-jqrk9\") pod \"opendatahub-operator-controller-manager-84df5575d5-jjjbs\" (UID: \"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.659966 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.659935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:36.814286 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:36.814265    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"]
Jul 06 22:52:36.817019 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:52:36.816991    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fddc9b8_8b37_4bb9_bd49_50aa0a5578fc.slice/crio-daf5fd83e4adab728ba925d4b27ac42897ad96936c502dccb6330946e98d1c63 WatchSource:0}: Error finding container daf5fd83e4adab728ba925d4b27ac42897ad96936c502dccb6330946e98d1c63: Status 404 returned error can't find the container with id daf5fd83e4adab728ba925d4b27ac42897ad96936c502dccb6330946e98d1c63
Jul 06 22:52:37.454921 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:37.454876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs" event={"ID":"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc","Type":"ContainerStarted","Data":"daf5fd83e4adab728ba925d4b27ac42897ad96936c502dccb6330946e98d1c63"}
Jul 06 22:52:40.469055 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:40.469025    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs" event={"ID":"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc","Type":"ContainerStarted","Data":"2efb354165028c6eb4d9cd24b8944f827202d0f090895d764d7c5a7032261bb7"}
Jul 06 22:52:41.473724 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:41.473688    2563 generic.go:356] "Generic (PLEG): container finished" podID="5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc" containerID="2efb354165028c6eb4d9cd24b8944f827202d0f090895d764d7c5a7032261bb7" exitCode=0
Jul 06 22:52:41.474299 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:41.473778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs" event={"ID":"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc","Type":"ContainerDied","Data":"2efb354165028c6eb4d9cd24b8944f827202d0f090895d764d7c5a7032261bb7"}
Jul 06 22:52:42.438733 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:42.438702    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5747d5c666-g6gkg"
Jul 06 22:52:42.480236 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:42.480206    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs" event={"ID":"5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc","Type":"ContainerStarted","Data":"945ea6c665a2dd9c12440dc67edbcf139ab79de866101056b2b3d4f69180b1c4"}
Jul 06 22:52:42.480744 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:42.480321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:52:42.502716 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:42.502666    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs" podStartSLOduration=1.92346686 podStartE2EDuration="6.502631022s" totalImagesPullingTime="3.543283495s" totalInitContainerRuntime="1.035880667s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:52:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:52:42.50099225 +0000 UTC m=+384.043323421" watchObservedRunningTime="2026-07-06 22:52:42.502631022 +0000 UTC m=+384.044962192"
Jul 06 22:52:53.485602 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:52:53.485572    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-jjjbs"
Jul 06 22:53:03.613722 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.613686    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"]
Jul 06 22:53:03.620596 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.620571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.623751 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.623726    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 22:53:03.623751 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.623742    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-66zt8\""
Jul 06 22:53:03.623933 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.623756    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 22:53:03.623992 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.623950    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 22:53:03.624051 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.624001    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 22:53:03.624103 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.624052    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 22:53:03.624259 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.624242    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 22:53:03.632889 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.632867    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"]
Jul 06 22:53:03.712415 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.712381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be26566-201d-448c-9f40-70a67e14c894-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.712415 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.712417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6be26566-201d-448c-9f40-70a67e14c894-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.712664 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.712453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6be26566-201d-448c-9f40-70a67e14c894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.712664 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.712532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be26566-201d-448c-9f40-70a67e14c894-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.712664 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.712576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6be26566-201d-448c-9f40-70a67e14c894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.712664 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.712594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6be26566-201d-448c-9f40-70a67e14c894-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.712664 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.712615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgj9g\" (UniqueName: \"kubernetes.io/projected/6be26566-201d-448c-9f40-70a67e14c894-kube-api-access-hgj9g\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.813985 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.813948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgj9g\" (UniqueName: \"kubernetes.io/projected/6be26566-201d-448c-9f40-70a67e14c894-kube-api-access-hgj9g\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.813985 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.813989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be26566-201d-448c-9f40-70a67e14c894-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.814202 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.814013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6be26566-201d-448c-9f40-70a67e14c894-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.814202 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.814074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6be26566-201d-448c-9f40-70a67e14c894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.814202 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.814108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be26566-201d-448c-9f40-70a67e14c894-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.814202 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.814153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6be26566-201d-448c-9f40-70a67e14c894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.814202 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.814176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6be26566-201d-448c-9f40-70a67e14c894-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.814860 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.814832    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6be26566-201d-448c-9f40-70a67e14c894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.816622 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.816592    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6be26566-201d-448c-9f40-70a67e14c894-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.816761 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.816687    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6be26566-201d-448c-9f40-70a67e14c894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.816874 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.816857    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be26566-201d-448c-9f40-70a67e14c894-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.816977 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.816961    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6be26566-201d-448c-9f40-70a67e14c894-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.823383 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.823354    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be26566-201d-448c-9f40-70a67e14c894-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.823564 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.823528    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgj9g\" (UniqueName: \"kubernetes.io/projected/6be26566-201d-448c-9f40-70a67e14c894-kube-api-access-hgj9g\") pod \"istiod-openshift-gateway-94bb8fbfd-pj6sf\" (UID: \"6be26566-201d-448c-9f40-70a67e14c894\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:03.929450 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:03.929420    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:04.088760 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:04.088735    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"]
Jul 06 22:53:04.090152 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:53:04.090127    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be26566_201d_448c_9f40_70a67e14c894.slice/crio-36b43cafde1d30b704562dde636bf8c4c6652a455cb2f31ed7ce2f15a1866efd WatchSource:0}: Error finding container 36b43cafde1d30b704562dde636bf8c4c6652a455cb2f31ed7ce2f15a1866efd: Status 404 returned error can't find the container with id 36b43cafde1d30b704562dde636bf8c4c6652a455cb2f31ed7ce2f15a1866efd
Jul 06 22:53:04.563071 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:04.563035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf" event={"ID":"6be26566-201d-448c-9f40-70a67e14c894","Type":"ContainerStarted","Data":"36b43cafde1d30b704562dde636bf8c4c6652a455cb2f31ed7ce2f15a1866efd"}
Jul 06 22:53:06.517321 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:06.517281    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 06 22:53:06.517544 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:06.517367    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 06 22:53:07.577049 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:07.577008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf" event={"ID":"6be26566-201d-448c-9f40-70a67e14c894","Type":"ContainerStarted","Data":"f186bab624e46369ca8d97e8978cd58672a91169194b3ff5b3acf2fcbcdd739d"}
Jul 06 22:53:07.577626 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:07.577199    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:07.578958 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:07.578935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf"
Jul 06 22:53:07.652817 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:53:07.652768    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pj6sf" podStartSLOduration=2.228425999 podStartE2EDuration="4.652750728s" totalImagesPullingTime="2.424324729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:07.649224986 +0000 UTC m=+409.191556156" watchObservedRunningTime="2026-07-06 22:53:07.652750728 +0000 UTC m=+409.195081897"
Jul 06 22:55:32.939301 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:32.939211    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"]
Jul 06 22:55:32.942652 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:32.942622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:32.945171 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:32.945146    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 22:55:32.945293 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:32.945147    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 22:55:32.946042 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:32.946020    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jmlxb\""
Jul 06 22:55:32.946042 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:32.946032    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 22:55:32.949863 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:32.949841    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"]
Jul 06 22:55:32.977984 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:32.977941    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtrl4\" (UniqueName: \"kubernetes.io/projected/17601543-6dd9-471d-a82c-4dbd4af89f3e-kube-api-access-rtrl4\") pod \"limitador-limitador-64c94fd64f-fpxsx\" (UID: \"17601543-6dd9-471d-a82c-4dbd4af89f3e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:32.977984 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:32.977992    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17601543-6dd9-471d-a82c-4dbd4af89f3e-config-file\") pod \"limitador-limitador-64c94fd64f-fpxsx\" (UID: \"17601543-6dd9-471d-a82c-4dbd4af89f3e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:33.039229 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:33.039194    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"]
Jul 06 22:55:33.078392 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:33.078348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17601543-6dd9-471d-a82c-4dbd4af89f3e-config-file\") pod \"limitador-limitador-64c94fd64f-fpxsx\" (UID: \"17601543-6dd9-471d-a82c-4dbd4af89f3e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:33.078595 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:33.078424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtrl4\" (UniqueName: \"kubernetes.io/projected/17601543-6dd9-471d-a82c-4dbd4af89f3e-kube-api-access-rtrl4\") pod \"limitador-limitador-64c94fd64f-fpxsx\" (UID: \"17601543-6dd9-471d-a82c-4dbd4af89f3e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:33.079074 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:33.079051    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17601543-6dd9-471d-a82c-4dbd4af89f3e-config-file\") pod \"limitador-limitador-64c94fd64f-fpxsx\" (UID: \"17601543-6dd9-471d-a82c-4dbd4af89f3e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:33.088712 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:33.088684    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtrl4\" (UniqueName: \"kubernetes.io/projected/17601543-6dd9-471d-a82c-4dbd4af89f3e-kube-api-access-rtrl4\") pod \"limitador-limitador-64c94fd64f-fpxsx\" (UID: \"17601543-6dd9-471d-a82c-4dbd4af89f3e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:33.253600 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:33.253507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:33.388138 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:33.388107    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"]
Jul 06 22:55:33.391405 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:55:33.391381    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17601543_6dd9_471d_a82c_4dbd4af89f3e.slice/crio-032a1b319e15c3f8a1f328a8f6f6ef554a6ad0ce0ad1ac13fe67ac4221626fb9 WatchSource:0}: Error finding container 032a1b319e15c3f8a1f328a8f6f6ef554a6ad0ce0ad1ac13fe67ac4221626fb9: Status 404 returned error can't find the container with id 032a1b319e15c3f8a1f328a8f6f6ef554a6ad0ce0ad1ac13fe67ac4221626fb9
Jul 06 22:55:34.147795 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:34.147722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx" event={"ID":"17601543-6dd9-471d-a82c-4dbd4af89f3e","Type":"ContainerStarted","Data":"032a1b319e15c3f8a1f328a8f6f6ef554a6ad0ce0ad1ac13fe67ac4221626fb9"}
Jul 06 22:55:37.162005 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:37.161974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx" event={"ID":"17601543-6dd9-471d-a82c-4dbd4af89f3e","Type":"ContainerStarted","Data":"41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2"}
Jul 06 22:55:37.162404 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:37.162168    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:37.177620 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:37.177564    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx" podStartSLOduration=2.413089821 podStartE2EDuration="5.177545765s" totalImagesPullingTime="2.764455944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:55:37.176983786 +0000 UTC m=+558.719314958" watchObservedRunningTime="2026-07-06 22:55:37.177545765 +0000 UTC m=+558.719876936"
Jul 06 22:55:47.883040 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:47.883001    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"]
Jul 06 22:55:47.883535 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:47.883215    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx" podUID="17601543-6dd9-471d-a82c-4dbd4af89f3e" containerName="limitador" containerID="cri-o://41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2" gracePeriod=30
Jul 06 22:55:47.883926 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:47.883822    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:48.418622 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:48.418592    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:48.486511 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:48.486410    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtrl4\" (UniqueName: \"kubernetes.io/projected/17601543-6dd9-471d-a82c-4dbd4af89f3e-kube-api-access-rtrl4\") pod \"17601543-6dd9-471d-a82c-4dbd4af89f3e\" (UID: \"17601543-6dd9-471d-a82c-4dbd4af89f3e\") "
Jul 06 22:55:48.486511 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:48.486453    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17601543-6dd9-471d-a82c-4dbd4af89f3e-config-file\") pod \"17601543-6dd9-471d-a82c-4dbd4af89f3e\" (UID: \"17601543-6dd9-471d-a82c-4dbd4af89f3e\") "
Jul 06 22:55:48.486855 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:48.486825    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17601543-6dd9-471d-a82c-4dbd4af89f3e-config-file" (OuterVolumeSpecName: "config-file") pod "17601543-6dd9-471d-a82c-4dbd4af89f3e" (UID: "17601543-6dd9-471d-a82c-4dbd4af89f3e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:55:48.488760 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:48.488736    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17601543-6dd9-471d-a82c-4dbd4af89f3e-kube-api-access-rtrl4" (OuterVolumeSpecName: "kube-api-access-rtrl4") pod "17601543-6dd9-471d-a82c-4dbd4af89f3e" (UID: "17601543-6dd9-471d-a82c-4dbd4af89f3e"). InnerVolumeSpecName "kube-api-access-rtrl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:55:48.587300 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:48.587263    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtrl4\" (UniqueName: \"kubernetes.io/projected/17601543-6dd9-471d-a82c-4dbd4af89f3e-kube-api-access-rtrl4\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:55:48.587300 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:48.587296    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17601543-6dd9-471d-a82c-4dbd4af89f3e-config-file\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:55:49.213566 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.213532    2563 generic.go:356] "Generic (PLEG): container finished" podID="17601543-6dd9-471d-a82c-4dbd4af89f3e" containerID="41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2" exitCode=0
Jul 06 22:55:49.214074 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.213587    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"
Jul 06 22:55:49.214074 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.213604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx" event={"ID":"17601543-6dd9-471d-a82c-4dbd4af89f3e","Type":"ContainerDied","Data":"41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2"}
Jul 06 22:55:49.214074 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.213664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fpxsx" event={"ID":"17601543-6dd9-471d-a82c-4dbd4af89f3e","Type":"ContainerDied","Data":"032a1b319e15c3f8a1f328a8f6f6ef554a6ad0ce0ad1ac13fe67ac4221626fb9"}
Jul 06 22:55:49.214074 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.213687    2563 scope.go:117] "RemoveContainer" containerID="41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2"
Jul 06 22:55:49.221847 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.221828    2563 scope.go:117] "RemoveContainer" containerID="41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2"
Jul 06 22:55:49.222161 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:55:49.222141    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2\": container with ID starting with 41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2 not found: ID does not exist" containerID="41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2"
Jul 06 22:55:49.222236 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.222172    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2"} err="failed to get container status \"41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2\": rpc error: code = NotFound desc = could not find container \"41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2\": container with ID starting with 41eddc8e72e550f233996776a3f76006281e37ab60067b957dacfcf50721dbc2 not found: ID does not exist"
Jul 06 22:55:49.230659 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.230621    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"]
Jul 06 22:55:49.234943 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.234918    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fpxsx"]
Jul 06 22:55:49.422209 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.422179    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"]
Jul 06 22:55:49.422492 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.422479    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17601543-6dd9-471d-a82c-4dbd4af89f3e" containerName="limitador"
Jul 06 22:55:49.422536 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.422495    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17601543-6dd9-471d-a82c-4dbd4af89f3e" containerName="limitador"
Jul 06 22:55:49.422573 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.422563    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="17601543-6dd9-471d-a82c-4dbd4af89f3e" containerName="limitador"
Jul 06 22:55:49.431412 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.431386    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:55:49.434269 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.434241    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bcjxj\""
Jul 06 22:55:49.434544 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.434521    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 06 22:55:49.434622 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.434563    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 06 22:55:49.434622 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.434529    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 06 22:55:49.434917 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.434885    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"]
Jul 06 22:55:49.494254 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.494150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a4dcc8aa-ba98-4a5e-963b-f3bb899e94da-data\") pod \"postgres-69fb4594cf-xvn66\" (UID: \"a4dcc8aa-ba98-4a5e-963b-f3bb899e94da\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:55:49.494254 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.494194    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpfmt\" (UniqueName: \"kubernetes.io/projected/a4dcc8aa-ba98-4a5e-963b-f3bb899e94da-kube-api-access-wpfmt\") pod \"postgres-69fb4594cf-xvn66\" (UID: \"a4dcc8aa-ba98-4a5e-963b-f3bb899e94da\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:55:49.595023 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.594987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a4dcc8aa-ba98-4a5e-963b-f3bb899e94da-data\") pod \"postgres-69fb4594cf-xvn66\" (UID: \"a4dcc8aa-ba98-4a5e-963b-f3bb899e94da\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:55:49.595023 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.595025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpfmt\" (UniqueName: \"kubernetes.io/projected/a4dcc8aa-ba98-4a5e-963b-f3bb899e94da-kube-api-access-wpfmt\") pod \"postgres-69fb4594cf-xvn66\" (UID: \"a4dcc8aa-ba98-4a5e-963b-f3bb899e94da\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:55:49.595395 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.595376    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a4dcc8aa-ba98-4a5e-963b-f3bb899e94da-data\") pod \"postgres-69fb4594cf-xvn66\" (UID: \"a4dcc8aa-ba98-4a5e-963b-f3bb899e94da\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:55:49.603066 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.603040    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpfmt\" (UniqueName: \"kubernetes.io/projected/a4dcc8aa-ba98-4a5e-963b-f3bb899e94da-kube-api-access-wpfmt\") pod \"postgres-69fb4594cf-xvn66\" (UID: \"a4dcc8aa-ba98-4a5e-963b-f3bb899e94da\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:55:49.744924 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.744831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:55:49.878594 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:49.878568    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"]
Jul 06 22:55:49.880845 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:55:49.880819    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dcc8aa_ba98_4a5e_963b_f3bb899e94da.slice/crio-a21139709dbd91c660ed8f5d95a0dcd742abee121a20cb049f2ce5f2ad8f1721 WatchSource:0}: Error finding container a21139709dbd91c660ed8f5d95a0dcd742abee121a20cb049f2ce5f2ad8f1721: Status 404 returned error can't find the container with id a21139709dbd91c660ed8f5d95a0dcd742abee121a20cb049f2ce5f2ad8f1721
Jul 06 22:55:50.219538 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:50.219495    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66" event={"ID":"a4dcc8aa-ba98-4a5e-963b-f3bb899e94da","Type":"ContainerStarted","Data":"a21139709dbd91c660ed8f5d95a0dcd742abee121a20cb049f2ce5f2ad8f1721"}
Jul 06 22:55:50.964526 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:50.964486    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17601543-6dd9-471d-a82c-4dbd4af89f3e" path="/var/lib/kubelet/pods/17601543-6dd9-471d-a82c-4dbd4af89f3e/volumes"
Jul 06 22:55:55.249144 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:55.249110    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66" event={"ID":"a4dcc8aa-ba98-4a5e-963b-f3bb899e94da","Type":"ContainerStarted","Data":"15dace2a9863f7538c254a87172281c1dd16344c4863dcb226234cafcc1f1c87"}
Jul 06 22:55:55.249656 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:55.249250    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:55:55.265342 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:55:55.265282    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66" podStartSLOduration=1.690906258 podStartE2EDuration="6.26526556s" totalImagesPullingTime="4.574359302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:55:55.26302248 +0000 UTC m=+576.805353652" watchObservedRunningTime="2026-07-06 22:55:55.26526556 +0000 UTC m=+576.807596733"
Jul 06 22:56:01.283092 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:01.283065    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xvn66"
Jul 06 22:56:08.334186 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.334152    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5df956d7f8-4ccpn"]
Jul 06 22:56:08.355737 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.355706    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5df956d7f8-4ccpn"]
Jul 06 22:56:08.355929 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.355817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:08.358593 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.358569    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 22:56:08.358827 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.358810    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mb2h7\""
Jul 06 22:56:08.358887 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.358858    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 22:56:08.454556 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.454515    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-webhook-cert\") pod \"maas-controller-5df956d7f8-4ccpn\" (UID: \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\") " pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:08.454816 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.454674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw4hz\" (UniqueName: \"kubernetes.io/projected/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-kube-api-access-dw4hz\") pod \"maas-controller-5df956d7f8-4ccpn\" (UID: \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\") " pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:08.497924 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.497892    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f58865b75-dhsr7"]
Jul 06 22:56:08.500096 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.500080    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:08.513072 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.513042    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f58865b75-dhsr7"]
Jul 06 22:56:08.555830 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.555790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzl6f\" (UniqueName: \"kubernetes.io/projected/1d298d5c-edb6-4363-a79d-37575f263612-kube-api-access-tzl6f\") pod \"maas-controller-5f58865b75-dhsr7\" (UID: \"1d298d5c-edb6-4363-a79d-37575f263612\") " pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:08.556011 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.555849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw4hz\" (UniqueName: \"kubernetes.io/projected/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-kube-api-access-dw4hz\") pod \"maas-controller-5df956d7f8-4ccpn\" (UID: \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\") " pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:08.556011 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.555894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-webhook-cert\") pod \"maas-controller-5df956d7f8-4ccpn\" (UID: \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\") " pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:08.556011 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.555918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d298d5c-edb6-4363-a79d-37575f263612-webhook-cert\") pod \"maas-controller-5f58865b75-dhsr7\" (UID: \"1d298d5c-edb6-4363-a79d-37575f263612\") " pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:08.558531 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.558505    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-webhook-cert\") pod \"maas-controller-5df956d7f8-4ccpn\" (UID: \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\") " pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:08.564339 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.564315    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw4hz\" (UniqueName: \"kubernetes.io/projected/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-kube-api-access-dw4hz\") pod \"maas-controller-5df956d7f8-4ccpn\" (UID: \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\") " pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:08.656937 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.656837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d298d5c-edb6-4363-a79d-37575f263612-webhook-cert\") pod \"maas-controller-5f58865b75-dhsr7\" (UID: \"1d298d5c-edb6-4363-a79d-37575f263612\") " pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:08.657105 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.656937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzl6f\" (UniqueName: \"kubernetes.io/projected/1d298d5c-edb6-4363-a79d-37575f263612-kube-api-access-tzl6f\") pod \"maas-controller-5f58865b75-dhsr7\" (UID: \"1d298d5c-edb6-4363-a79d-37575f263612\") " pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:08.660632 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.660592    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d298d5c-edb6-4363-a79d-37575f263612-webhook-cert\") pod \"maas-controller-5f58865b75-dhsr7\" (UID: \"1d298d5c-edb6-4363-a79d-37575f263612\") " pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:08.665287 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.665257    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzl6f\" (UniqueName: \"kubernetes.io/projected/1d298d5c-edb6-4363-a79d-37575f263612-kube-api-access-tzl6f\") pod \"maas-controller-5f58865b75-dhsr7\" (UID: \"1d298d5c-edb6-4363-a79d-37575f263612\") " pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:08.667093 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.667074    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:08.795251 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.795220    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5df956d7f8-4ccpn"]
Jul 06 22:56:08.798091 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:56:08.798067    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dee4d32_dcb5_4d48_8d7e_af1fbfd5d7a4.slice/crio-5541e7aab2a3cfc8b167cd6e856a69f671e3c647409e92d43ac5dbe02f8b5b9d WatchSource:0}: Error finding container 5541e7aab2a3cfc8b167cd6e856a69f671e3c647409e92d43ac5dbe02f8b5b9d: Status 404 returned error can't find the container with id 5541e7aab2a3cfc8b167cd6e856a69f671e3c647409e92d43ac5dbe02f8b5b9d
Jul 06 22:56:08.810753 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.810733    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:08.937948 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:08.937922    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f58865b75-dhsr7"]
Jul 06 22:56:08.940259 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:56:08.940236    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d298d5c_edb6_4363_a79d_37575f263612.slice/crio-61391471cb630324fbd2c96fecd16bd98b1edeba63b43337f9d08cc0aa94e0a0 WatchSource:0}: Error finding container 61391471cb630324fbd2c96fecd16bd98b1edeba63b43337f9d08cc0aa94e0a0: Status 404 returned error can't find the container with id 61391471cb630324fbd2c96fecd16bd98b1edeba63b43337f9d08cc0aa94e0a0
Jul 06 22:56:09.307253 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:09.307160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f58865b75-dhsr7" event={"ID":"1d298d5c-edb6-4363-a79d-37575f263612","Type":"ContainerStarted","Data":"61391471cb630324fbd2c96fecd16bd98b1edeba63b43337f9d08cc0aa94e0a0"}
Jul 06 22:56:09.308227 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:09.308202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5df956d7f8-4ccpn" event={"ID":"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4","Type":"ContainerStarted","Data":"5541e7aab2a3cfc8b167cd6e856a69f671e3c647409e92d43ac5dbe02f8b5b9d"}
Jul 06 22:56:11.326143 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:11.326100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f58865b75-dhsr7" event={"ID":"1d298d5c-edb6-4363-a79d-37575f263612","Type":"ContainerStarted","Data":"b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c"}
Jul 06 22:56:11.326617 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:11.326209    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:11.327850 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:11.327826    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5df956d7f8-4ccpn" event={"ID":"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4","Type":"ContainerStarted","Data":"ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e"}
Jul 06 22:56:11.327984 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:11.327962    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:11.345280 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:11.345218    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f58865b75-dhsr7" podStartSLOduration=1.399919885 podStartE2EDuration="3.345198324s" totalImagesPullingTime="1.945278439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:11.343910606 +0000 UTC m=+592.886241777" watchObservedRunningTime="2026-07-06 22:56:11.345198324 +0000 UTC m=+592.887529493"
Jul 06 22:56:11.363738 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:11.363686    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5df956d7f8-4ccpn" podStartSLOduration=1.285909311 podStartE2EDuration="3.363667025s" totalImagesPullingTime="2.077757714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:11.363121985 +0000 UTC m=+592.905453155" watchObservedRunningTime="2026-07-06 22:56:11.363667025 +0000 UTC m=+592.905998188"
Jul 06 22:56:22.338171 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.338135    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 22:56:22.338670 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.338222    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:22.396356 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.396311    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5df956d7f8-4ccpn"]
Jul 06 22:56:22.396575 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.396548    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5df956d7f8-4ccpn" podUID="6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4" containerName="manager" containerID="cri-o://ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e" gracePeriod=30
Jul 06 22:56:22.637909 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.637882    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:22.786970 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.786931    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw4hz\" (UniqueName: \"kubernetes.io/projected/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-kube-api-access-dw4hz\") pod \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\" (UID: \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\") "
Jul 06 22:56:22.787170 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.786987    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-webhook-cert\") pod \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\" (UID: \"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4\") "
Jul 06 22:56:22.789307 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.789275    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4" (UID: "6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:56:22.789307 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.789293    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-kube-api-access-dw4hz" (OuterVolumeSpecName: "kube-api-access-dw4hz") pod "6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4" (UID: "6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4"). InnerVolumeSpecName "kube-api-access-dw4hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:56:22.888125 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.888014    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw4hz\" (UniqueName: \"kubernetes.io/projected/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-kube-api-access-dw4hz\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:56:22.888125 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:22.888049    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4-webhook-cert\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 22:56:23.378369 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:23.378332    2563 generic.go:356] "Generic (PLEG): container finished" podID="6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4" containerID="ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e" exitCode=0
Jul 06 22:56:23.378831 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:23.378386    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5df956d7f8-4ccpn"
Jul 06 22:56:23.378831 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:23.378400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5df956d7f8-4ccpn" event={"ID":"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4","Type":"ContainerDied","Data":"ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e"}
Jul 06 22:56:23.378831 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:23.378458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5df956d7f8-4ccpn" event={"ID":"6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4","Type":"ContainerDied","Data":"5541e7aab2a3cfc8b167cd6e856a69f671e3c647409e92d43ac5dbe02f8b5b9d"}
Jul 06 22:56:23.378831 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:23.378480    2563 scope.go:117] "RemoveContainer" containerID="ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e"
Jul 06 22:56:23.386836 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:23.386816    2563 scope.go:117] "RemoveContainer" containerID="ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e"
Jul 06 22:56:23.387137 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:56:23.387112    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e\": container with ID starting with ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e not found: ID does not exist" containerID="ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e"
Jul 06 22:56:23.387185 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:23.387148    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e"} err="failed to get container status \"ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e\": rpc error: code = NotFound desc = could not find container \"ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e\": container with ID starting with ccac26ab4edb43f53aefdae2928f4ae7ecd79e77150b9a9895b2457490c75f1e not found: ID does not exist"
Jul 06 22:56:23.393518 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:23.393490    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5df956d7f8-4ccpn"]
Jul 06 22:56:23.395592 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:23.395563    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5df956d7f8-4ccpn"]
Jul 06 22:56:24.962718 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:24.962677    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4" path="/var/lib/kubelet/pods/6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4/volumes"
Jul 06 22:56:43.470844 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.470760    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"]
Jul 06 22:56:43.471309 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.471095    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4" containerName="manager"
Jul 06 22:56:43.471309 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.471107    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4" containerName="manager"
Jul 06 22:56:43.471309 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.471161    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dee4d32-dcb5-4d48-8d7e-af1fbfd5d7a4" containerName="manager"
Jul 06 22:56:43.473936 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.473916    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:43.476368 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.476333    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 06 22:56:43.476526 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.476392    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhcjn\""
Jul 06 22:56:43.483917 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.483885    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"]
Jul 06 22:56:43.539554 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.539515    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-maas-api-tls\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:43.539753 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.539570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kn2g\" (UniqueName: \"kubernetes.io/projected/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-kube-api-access-8kn2g\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:43.539753 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.539630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-openshift-service-ca\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:43.640127 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.640085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kn2g\" (UniqueName: \"kubernetes.io/projected/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-kube-api-access-8kn2g\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:43.640331 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.640137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-openshift-service-ca\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:43.640331 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.640180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-maas-api-tls\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:43.640331 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:56:43.640310    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 06 22:56:43.640495 ip-10-0-140-53 kubenswrapper[2563]: E0706 22:56:43.640401    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-maas-api-tls podName:a0f09cde-6e3d-46b4-ba15-fb194e102c7a nodeName:}" failed. No retries permitted until 2026-07-06 22:56:44.140367068 +0000 UTC m=+625.682698221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-maas-api-tls") pod "maas-api-578456fb8d-4cdp5" (UID: "a0f09cde-6e3d-46b4-ba15-fb194e102c7a") : secret "maas-api-serving-cert" not found
Jul 06 22:56:43.640829 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.640808    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-openshift-service-ca\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:43.649056 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.649029    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kn2g\" (UniqueName: \"kubernetes.io/projected/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-kube-api-access-8kn2g\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:43.941179 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.941136    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5rm74"]
Jul 06 22:56:43.944346 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.944321    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:56:43.946815 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.946793    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 22:56:43.946946 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.946852    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nrgpx\""
Jul 06 22:56:43.954149 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:43.954122    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5rm74"]
Jul 06 22:56:44.043464 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.043428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b90e7886-4b88-4923-a310-61d4c20d28e8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5rm74\" (UID: \"b90e7886-4b88-4923-a310-61d4c20d28e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:56:44.043464 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.043470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpvrs\" (UniqueName: \"kubernetes.io/projected/b90e7886-4b88-4923-a310-61d4c20d28e8-kube-api-access-fpvrs\") pod \"payload-pre-processing-7964457696-5rm74\" (UID: \"b90e7886-4b88-4923-a310-61d4c20d28e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:56:44.144159 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.144123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b90e7886-4b88-4923-a310-61d4c20d28e8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5rm74\" (UID: \"b90e7886-4b88-4923-a310-61d4c20d28e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:56:44.144159 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.144160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpvrs\" (UniqueName: \"kubernetes.io/projected/b90e7886-4b88-4923-a310-61d4c20d28e8-kube-api-access-fpvrs\") pod \"payload-pre-processing-7964457696-5rm74\" (UID: \"b90e7886-4b88-4923-a310-61d4c20d28e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:56:44.144374 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.144213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-maas-api-tls\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:44.144981 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.144960    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b90e7886-4b88-4923-a310-61d4c20d28e8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-5rm74\" (UID: \"b90e7886-4b88-4923-a310-61d4c20d28e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:56:44.146831 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.146801    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0f09cde-6e3d-46b4-ba15-fb194e102c7a-maas-api-tls\") pod \"maas-api-578456fb8d-4cdp5\" (UID: \"a0f09cde-6e3d-46b4-ba15-fb194e102c7a\") " pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:44.154951 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.154926    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpvrs\" (UniqueName: \"kubernetes.io/projected/b90e7886-4b88-4923-a310-61d4c20d28e8-kube-api-access-fpvrs\") pod \"payload-pre-processing-7964457696-5rm74\" (UID: \"b90e7886-4b88-4923-a310-61d4c20d28e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:56:44.255784 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.255699    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:56:44.384873 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.384839    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:44.393129 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.393104    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-5rm74"]
Jul 06 22:56:44.397176 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:56:44.397149    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb90e7886_4b88_4923_a310_61d4c20d28e8.slice/crio-aa29585fe5f01d7120ae4b8d005f3b764268fc43079cfc64cbb2f64a137fc134 WatchSource:0}: Error finding container aa29585fe5f01d7120ae4b8d005f3b764268fc43079cfc64cbb2f64a137fc134: Status 404 returned error can't find the container with id aa29585fe5f01d7120ae4b8d005f3b764268fc43079cfc64cbb2f64a137fc134
Jul 06 22:56:44.467722 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.467673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5rm74" event={"ID":"b90e7886-4b88-4923-a310-61d4c20d28e8","Type":"ContainerStarted","Data":"aa29585fe5f01d7120ae4b8d005f3b764268fc43079cfc64cbb2f64a137fc134"}
Jul 06 22:56:44.521282 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:44.521252    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"]
Jul 06 22:56:44.523478 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:56:44.523445    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f09cde_6e3d_46b4_ba15_fb194e102c7a.slice/crio-5bffe3d29b3f9aaa3def8580ef38ff361c39c0151ee9cef3c03398c0ed0c0e92 WatchSource:0}: Error finding container 5bffe3d29b3f9aaa3def8580ef38ff361c39c0151ee9cef3c03398c0ed0c0e92: Status 404 returned error can't find the container with id 5bffe3d29b3f9aaa3def8580ef38ff361c39c0151ee9cef3c03398c0ed0c0e92
Jul 06 22:56:45.474968 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:45.474898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5" event={"ID":"a0f09cde-6e3d-46b4-ba15-fb194e102c7a","Type":"ContainerStarted","Data":"5bffe3d29b3f9aaa3def8580ef38ff361c39c0151ee9cef3c03398c0ed0c0e92"}
Jul 06 22:56:46.481735 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:46.481684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-5rm74" event={"ID":"b90e7886-4b88-4923-a310-61d4c20d28e8","Type":"ContainerStarted","Data":"a19ebe9fff3c9d1bad6fadd68a2475dc0829bf326e82390c5ee282250a12939a"}
Jul 06 22:56:46.482227 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:46.481827    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:56:46.483347 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:46.483327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5" event={"ID":"a0f09cde-6e3d-46b4-ba15-fb194e102c7a","Type":"ContainerStarted","Data":"3cad46ddd0691b82f84c9ef9bdf0497bc869011ac9ea20e6f997136f10716a91"}
Jul 06 22:56:46.483475 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:46.483458    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:46.498193 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:46.498141    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-5rm74" podStartSLOduration=1.7905540549999999 podStartE2EDuration="3.498124948s" totalImagesPullingTime="1.707570893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:46.497296781 +0000 UTC m=+628.039627951" watchObservedRunningTime="2026-07-06 22:56:46.498124948 +0000 UTC m=+628.040456120"
Jul 06 22:56:46.514363 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:46.514317    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5" podStartSLOduration=1.930300001 podStartE2EDuration="3.514299595s" totalImagesPullingTime="1.583999594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:46.512180706 +0000 UTC m=+628.054511876" watchObservedRunningTime="2026-07-06 22:56:46.514299595 +0000 UTC m=+628.056630765"
Jul 06 22:56:52.493965 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:52.493933    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-578456fb8d-4cdp5"
Jul 06 22:56:57.494596 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:56:57.494567    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-5rm74"
Jul 06 22:57:31.966081 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:31.966046    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"]
Jul 06 22:57:31.970813 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:31.970789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:31.973521 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:31.973497    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 06 22:57:31.973671 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:31.973554    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 22:57:31.974534 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:31.974512    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 22:57:31.974623 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:31.974535    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bs2s8\""
Jul 06 22:57:31.982908 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:31.982847    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"]
Jul 06 22:57:32.155038 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.155000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.155038 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.155046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c380c5d0-99cb-4295-a43a-22ba73d4557b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.155308 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.155070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.155308 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.155085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.155308 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.155212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.155308 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.155259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwrx5\" (UniqueName: \"kubernetes.io/projected/c380c5d0-99cb-4295-a43a-22ba73d4557b-kube-api-access-nwrx5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.256426 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.256324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.256426 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.256377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c380c5d0-99cb-4295-a43a-22ba73d4557b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.256703 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.256506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.256703 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.256552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.256703 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.256626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.256852 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.256702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwrx5\" (UniqueName: \"kubernetes.io/projected/c380c5d0-99cb-4295-a43a-22ba73d4557b-kube-api-access-nwrx5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.256965 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.256920    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.256965 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.256937    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.257318 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.257019    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.258886 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.258863    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c380c5d0-99cb-4295-a43a-22ba73d4557b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.259036 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.259020    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c380c5d0-99cb-4295-a43a-22ba73d4557b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.264745 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.264718    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwrx5\" (UniqueName: \"kubernetes.io/projected/c380c5d0-99cb-4295-a43a-22ba73d4557b-kube-api-access-nwrx5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h\" (UID: \"c380c5d0-99cb-4295-a43a-22ba73d4557b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.283814 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.283774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:32.421540 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.421502    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"]
Jul 06 22:57:32.424586 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:57:32.424560    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc380c5d0_99cb_4295_a43a_22ba73d4557b.slice/crio-ea85a390e9d132e51569f34652102869d103855af7f8d225cf03b96dfaa42759 WatchSource:0}: Error finding container ea85a390e9d132e51569f34652102869d103855af7f8d225cf03b96dfaa42759: Status 404 returned error can't find the container with id ea85a390e9d132e51569f34652102869d103855af7f8d225cf03b96dfaa42759
Jul 06 22:57:32.426020 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.425999    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:57:32.672944 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:32.672913    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h" event={"ID":"c380c5d0-99cb-4295-a43a-22ba73d4557b","Type":"ContainerStarted","Data":"ea85a390e9d132e51569f34652102869d103855af7f8d225cf03b96dfaa42759"}
Jul 06 22:57:33.678956 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:33.678918    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h" event={"ID":"c380c5d0-99cb-4295-a43a-22ba73d4557b","Type":"ContainerStarted","Data":"99ead7e6941b09a2a9bc870f85aef9e3aed14fbdd1af0bf60e8ad9ee373f9f60"}
Jul 06 22:57:33.696601 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:33.696485    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h" podStartSLOduration=1.674951376 podStartE2EDuration="2.696467318s" totalImagesPullingTime="1.021515942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:57:33.695208991 +0000 UTC m=+675.237540171" watchObservedRunningTime="2026-07-06 22:57:33.696467318 +0000 UTC m=+675.238798488"
Jul 06 22:57:42.283996 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:42.283958    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:42.284597 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:42.284293    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:42.297013 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:42.296986    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:57:42.728508 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:57:42.728480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h"
Jul 06 22:58:10.278153 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.278071    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"]
Jul 06 22:58:10.284218 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.284182    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.286808 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.286776    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 06 22:58:10.291783 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.291564    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"]
Jul 06 22:58:10.360569 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.360535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da761aac-d953-4196-8661-15fa1e055977-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.360809 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.360578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.360809 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.360605    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.360809 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.360700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bvvt\" (UniqueName: \"kubernetes.io/projected/da761aac-d953-4196-8661-15fa1e055977-kube-api-access-5bvvt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.360809 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.360777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.360984 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.360834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.461695 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.461657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bvvt\" (UniqueName: \"kubernetes.io/projected/da761aac-d953-4196-8661-15fa1e055977-kube-api-access-5bvvt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.461899 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.461705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.461971 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.461921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.462039 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.461992    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da761aac-d953-4196-8661-15fa1e055977-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.462101 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.462034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.462101 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.462060    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.462204 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.462129    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.462303 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.462272    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.462420 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.462381    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.464494 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.464468    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da761aac-d953-4196-8661-15fa1e055977-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.464740 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.464720    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da761aac-d953-4196-8661-15fa1e055977-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.470084 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.470050    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bvvt\" (UniqueName: \"kubernetes.io/projected/da761aac-d953-4196-8661-15fa1e055977-kube-api-access-5bvvt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b\" (UID: \"da761aac-d953-4196-8661-15fa1e055977\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.596799 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.596705    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:10.732734 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.732708    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"]
Jul 06 22:58:10.734889 ip-10-0-140-53 kubenswrapper[2563]: W0706 22:58:10.734856    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda761aac_d953_4196_8661_15fa1e055977.slice/crio-f227caef86fa9970bb69dde2d4e9b4a5ebeaefa9e51270c747b5efc221c1d097 WatchSource:0}: Error finding container f227caef86fa9970bb69dde2d4e9b4a5ebeaefa9e51270c747b5efc221c1d097: Status 404 returned error can't find the container with id f227caef86fa9970bb69dde2d4e9b4a5ebeaefa9e51270c747b5efc221c1d097
Jul 06 22:58:10.832792 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:10.832761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b" event={"ID":"da761aac-d953-4196-8661-15fa1e055977","Type":"ContainerStarted","Data":"f227caef86fa9970bb69dde2d4e9b4a5ebeaefa9e51270c747b5efc221c1d097"}
Jul 06 22:58:11.838251 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:11.838213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b" event={"ID":"da761aac-d953-4196-8661-15fa1e055977","Type":"ContainerStarted","Data":"950af9bccae20676bc3d63d14b9cf8044a4b951a03a06e2f8324fdc03ae45274"}
Jul 06 22:58:20.597289 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:20.597247    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:20.597821 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:20.597606    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:20.609958 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:20.609930    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 22:58:20.627200 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:20.627151    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b" podStartSLOduration=10.452888255 podStartE2EDuration="10.627134447s" totalImagesPullingTime="174.246192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:58:11.856005282 +0000 UTC m=+713.398336454" watchObservedRunningTime="2026-07-06 22:58:20.627134447 +0000 UTC m=+722.169465617"
Jul 06 22:58:20.887052 ip-10-0-140-53 kubenswrapper[2563]: I0706 22:58:20.886970    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b"
Jul 06 23:00:32.904509 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:32.904465    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f58865b75-dhsr7"]
Jul 06 23:00:32.905070 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:32.904734    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5f58865b75-dhsr7" podUID="1d298d5c-edb6-4363-a79d-37575f263612" containerName="manager" containerID="cri-o://b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c" gracePeriod=30
Jul 06 23:00:33.158624 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.158543    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 23:00:33.268693 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.268624    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzl6f\" (UniqueName: \"kubernetes.io/projected/1d298d5c-edb6-4363-a79d-37575f263612-kube-api-access-tzl6f\") pod \"1d298d5c-edb6-4363-a79d-37575f263612\" (UID: \"1d298d5c-edb6-4363-a79d-37575f263612\") "
Jul 06 23:00:33.268879 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.268706    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d298d5c-edb6-4363-a79d-37575f263612-webhook-cert\") pod \"1d298d5c-edb6-4363-a79d-37575f263612\" (UID: \"1d298d5c-edb6-4363-a79d-37575f263612\") "
Jul 06 23:00:33.271031 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.270990    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d298d5c-edb6-4363-a79d-37575f263612-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1d298d5c-edb6-4363-a79d-37575f263612" (UID: "1d298d5c-edb6-4363-a79d-37575f263612"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:00:33.271143 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.270998    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d298d5c-edb6-4363-a79d-37575f263612-kube-api-access-tzl6f" (OuterVolumeSpecName: "kube-api-access-tzl6f") pod "1d298d5c-edb6-4363-a79d-37575f263612" (UID: "1d298d5c-edb6-4363-a79d-37575f263612"). InnerVolumeSpecName "kube-api-access-tzl6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:00:33.369604 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.369562    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzl6f\" (UniqueName: \"kubernetes.io/projected/1d298d5c-edb6-4363-a79d-37575f263612-kube-api-access-tzl6f\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:00:33.369604 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.369595    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d298d5c-edb6-4363-a79d-37575f263612-webhook-cert\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:00:33.446486 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.446450    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d298d5c-edb6-4363-a79d-37575f263612" containerID="b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c" exitCode=0
Jul 06 23:00:33.446680 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.446510    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f58865b75-dhsr7"
Jul 06 23:00:33.446680 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.446537    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f58865b75-dhsr7" event={"ID":"1d298d5c-edb6-4363-a79d-37575f263612","Type":"ContainerDied","Data":"b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c"}
Jul 06 23:00:33.446680 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.446609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f58865b75-dhsr7" event={"ID":"1d298d5c-edb6-4363-a79d-37575f263612","Type":"ContainerDied","Data":"61391471cb630324fbd2c96fecd16bd98b1edeba63b43337f9d08cc0aa94e0a0"}
Jul 06 23:00:33.446680 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.446632    2563 scope.go:117] "RemoveContainer" containerID="b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c"
Jul 06 23:00:33.456183 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.456162    2563 scope.go:117] "RemoveContainer" containerID="b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c"
Jul 06 23:00:33.456448 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:00:33.456428    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c\": container with ID starting with b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c not found: ID does not exist" containerID="b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c"
Jul 06 23:00:33.456505 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.456457    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c"} err="failed to get container status \"b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c\": rpc error: code = NotFound desc = could not find container \"b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c\": container with ID starting with b0855ed3188662b23909365457f9591ba7c054526c91e1ad7c0329e8316d353c not found: ID does not exist"
Jul 06 23:00:33.466819 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.466789    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f58865b75-dhsr7"]
Jul 06 23:00:33.470654 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:33.470613    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f58865b75-dhsr7"]
Jul 06 23:00:34.218006 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.217970    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f58865b75-l95kx"]
Jul 06 23:00:34.218409 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.218274    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d298d5c-edb6-4363-a79d-37575f263612" containerName="manager"
Jul 06 23:00:34.218409 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.218285    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d298d5c-edb6-4363-a79d-37575f263612" containerName="manager"
Jul 06 23:00:34.218409 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.218340    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d298d5c-edb6-4363-a79d-37575f263612" containerName="manager"
Jul 06 23:00:34.222460 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.222434    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:00:34.224930 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.224903    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 23:00:34.225874 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.225852    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mb2h7\""
Jul 06 23:00:34.225989 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.225895    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 23:00:34.230117 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.230089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f58865b75-l95kx"]
Jul 06 23:00:34.276139 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.276100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szdkr\" (UniqueName: \"kubernetes.io/projected/a08adfff-5ed0-4ea7-8bd6-d907bab56828-kube-api-access-szdkr\") pod \"maas-controller-5f58865b75-l95kx\" (UID: \"a08adfff-5ed0-4ea7-8bd6-d907bab56828\") " pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:00:34.276139 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.276138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a08adfff-5ed0-4ea7-8bd6-d907bab56828-webhook-cert\") pod \"maas-controller-5f58865b75-l95kx\" (UID: \"a08adfff-5ed0-4ea7-8bd6-d907bab56828\") " pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:00:34.376785 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.376743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szdkr\" (UniqueName: \"kubernetes.io/projected/a08adfff-5ed0-4ea7-8bd6-d907bab56828-kube-api-access-szdkr\") pod \"maas-controller-5f58865b75-l95kx\" (UID: \"a08adfff-5ed0-4ea7-8bd6-d907bab56828\") " pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:00:34.376785 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.376788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a08adfff-5ed0-4ea7-8bd6-d907bab56828-webhook-cert\") pod \"maas-controller-5f58865b75-l95kx\" (UID: \"a08adfff-5ed0-4ea7-8bd6-d907bab56828\") " pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:00:34.379331 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.379305    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a08adfff-5ed0-4ea7-8bd6-d907bab56828-webhook-cert\") pod \"maas-controller-5f58865b75-l95kx\" (UID: \"a08adfff-5ed0-4ea7-8bd6-d907bab56828\") " pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:00:34.385349 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.385312    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szdkr\" (UniqueName: \"kubernetes.io/projected/a08adfff-5ed0-4ea7-8bd6-d907bab56828-kube-api-access-szdkr\") pod \"maas-controller-5f58865b75-l95kx\" (UID: \"a08adfff-5ed0-4ea7-8bd6-d907bab56828\") " pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:00:34.534777 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.534669    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:00:34.671758 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.671728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f58865b75-l95kx"]
Jul 06 23:00:34.674137 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:00:34.674107    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08adfff_5ed0_4ea7_8bd6_d907bab56828.slice/crio-9d9aba370d40f98027a0b3d6606f147d1203e5b936e397154e11cfcca8fb4275 WatchSource:0}: Error finding container 9d9aba370d40f98027a0b3d6606f147d1203e5b936e397154e11cfcca8fb4275: Status 404 returned error can't find the container with id 9d9aba370d40f98027a0b3d6606f147d1203e5b936e397154e11cfcca8fb4275
Jul 06 23:00:34.963491 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:34.963455    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d298d5c-edb6-4363-a79d-37575f263612" path="/var/lib/kubelet/pods/1d298d5c-edb6-4363-a79d-37575f263612/volumes"
Jul 06 23:00:35.457214 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:35.457173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f58865b75-l95kx" event={"ID":"a08adfff-5ed0-4ea7-8bd6-d907bab56828","Type":"ContainerStarted","Data":"9d9aba370d40f98027a0b3d6606f147d1203e5b936e397154e11cfcca8fb4275"}
Jul 06 23:00:36.464155 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:36.464115    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f58865b75-l95kx" event={"ID":"a08adfff-5ed0-4ea7-8bd6-d907bab56828","Type":"ContainerStarted","Data":"c0b2928f0aa4e7d45bf5c401ecd65467f5f068f7b0ac622854b7be4a9586cf92"}
Jul 06 23:00:36.464600 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:36.464203    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:00:36.480715 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:36.480599    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f58865b75-l95kx" podStartSLOduration=0.772493908 podStartE2EDuration="2.48058108s" totalImagesPullingTime="1.708087172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:00:36.479622544 +0000 UTC m=+858.021953713" watchObservedRunningTime="2026-07-06 23:00:36.48058108 +0000 UTC m=+858.022912250"
Jul 06 23:00:47.474585 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:00:47.474555    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f58865b75-l95kx"
Jul 06 23:12:36.810192 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.810146    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"]
Jul 06 23:12:36.813808 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.813781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:36.817093 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.817072    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-371df82c-serving-cert\""
Jul 06 23:12:36.826479 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.826437    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"]
Jul 06 23:12:36.890169 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.890129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrwh5\" (UniqueName: \"kubernetes.io/projected/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-kube-api-access-mrwh5\") pod \"maas-api-e2e-ait-371df82c-798944cb87-6752p\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:36.890363 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.890182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-maas-api-tls\") pod \"maas-api-e2e-ait-371df82c-798944cb87-6752p\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:36.890363 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.890248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-openshift-service-ca\") pod \"maas-api-e2e-ait-371df82c-798944cb87-6752p\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:36.991348 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.991308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrwh5\" (UniqueName: \"kubernetes.io/projected/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-kube-api-access-mrwh5\") pod \"maas-api-e2e-ait-371df82c-798944cb87-6752p\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:36.991524 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.991356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-maas-api-tls\") pod \"maas-api-e2e-ait-371df82c-798944cb87-6752p\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:36.991524 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.991478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-openshift-service-ca\") pod \"maas-api-e2e-ait-371df82c-798944cb87-6752p\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:36.992151 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.992119    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-openshift-service-ca\") pod \"maas-api-e2e-ait-371df82c-798944cb87-6752p\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:36.994925 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:36.994897    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-maas-api-tls\") pod \"maas-api-e2e-ait-371df82c-798944cb87-6752p\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:37.001153 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:37.001117    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrwh5\" (UniqueName: \"kubernetes.io/projected/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-kube-api-access-mrwh5\") pod \"maas-api-e2e-ait-371df82c-798944cb87-6752p\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:37.126345 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:37.126257    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:37.282231 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:37.282136    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"]
Jul 06 23:12:37.284701 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:12:37.284673    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cdc07d6_73d8_4ce9_b337_82ab2aaa78a8.slice/crio-18446b4558e13d99a9d12116e88796c6e7eeabd84cac927cf5e7e746eb2ce21f WatchSource:0}: Error finding container 18446b4558e13d99a9d12116e88796c6e7eeabd84cac927cf5e7e746eb2ce21f: Status 404 returned error can't find the container with id 18446b4558e13d99a9d12116e88796c6e7eeabd84cac927cf5e7e746eb2ce21f
Jul 06 23:12:37.286110 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:37.286090    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 23:12:37.530160 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:37.530125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p" event={"ID":"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8","Type":"ContainerStarted","Data":"18446b4558e13d99a9d12116e88796c6e7eeabd84cac927cf5e7e746eb2ce21f"}
Jul 06 23:12:38.536152 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:38.536112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p" event={"ID":"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8","Type":"ContainerStarted","Data":"d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52"}
Jul 06 23:12:38.536564 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:38.536219    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:42.056676 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.056582    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p" podStartSLOduration=5.355874496 podStartE2EDuration="6.056559076s" totalImagesPullingTime="700.68458ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:12:38.57145219 +0000 UTC m=+1580.113783360" watchObservedRunningTime="2026-07-06 23:12:42.056559076 +0000 UTC m=+1583.598890250"
Jul 06 23:12:42.058245 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.058192    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"]
Jul 06 23:12:42.059255 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.058707    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p" podUID="3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8" containerName="maas-api" containerID="cri-o://d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52" gracePeriod=30
Jul 06 23:12:42.325102 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.325072    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:42.442070 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.442032    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-maas-api-tls\") pod \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") "
Jul 06 23:12:42.442295 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.442097    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-openshift-service-ca\") pod \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") "
Jul 06 23:12:42.442295 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.442185    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrwh5\" (UniqueName: \"kubernetes.io/projected/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-kube-api-access-mrwh5\") pod \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\" (UID: \"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8\") "
Jul 06 23:12:42.442559 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.442524    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8" (UID: "3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:12:42.444674 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.444619    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8" (UID: "3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:12:42.444797 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.444668    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-kube-api-access-mrwh5" (OuterVolumeSpecName: "kube-api-access-mrwh5") pod "3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8" (UID: "3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8"). InnerVolumeSpecName "kube-api-access-mrwh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:12:42.543556 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.543518    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:12:42.543556 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.543547    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:12:42.543556 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.543556    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrwh5\" (UniqueName: \"kubernetes.io/projected/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8-kube-api-access-mrwh5\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:12:42.553908 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.553868    2563 generic.go:356] "Generic (PLEG): container finished" podID="3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8" containerID="d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52" exitCode=0
Jul 06 23:12:42.554101 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.553916    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"
Jul 06 23:12:42.554101 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.553959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p" event={"ID":"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8","Type":"ContainerDied","Data":"d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52"}
Jul 06 23:12:42.554101 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.554020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p" event={"ID":"3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8","Type":"ContainerDied","Data":"18446b4558e13d99a9d12116e88796c6e7eeabd84cac927cf5e7e746eb2ce21f"}
Jul 06 23:12:42.554101 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.554049    2563 scope.go:117] "RemoveContainer" containerID="d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52"
Jul 06 23:12:42.565463 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.565443    2563 scope.go:117] "RemoveContainer" containerID="d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52"
Jul 06 23:12:42.565887 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:12:42.565865    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52\": container with ID starting with d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52 not found: ID does not exist" containerID="d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52"
Jul 06 23:12:42.565952 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.565895    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52"} err="failed to get container status \"d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52\": rpc error: code = NotFound desc = could not find container \"d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52\": container with ID starting with d1edcf472b56a88ea10eac67398d831a177b0bef8b0b72f37a6d311b3a039a52 not found: ID does not exist"
Jul 06 23:12:42.584670 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:12:42.579275    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cdc07d6_73d8_4ce9_b337_82ab2aaa78a8.slice\": RecentStats: unable to find data in memory cache]"
Jul 06 23:12:42.585363 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.585322    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"]
Jul 06 23:12:42.586157 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.586134    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-371df82c-798944cb87-6752p"]
Jul 06 23:12:42.963337 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:42.963285    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8" path="/var/lib/kubelet/pods/3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8/volumes"
Jul 06 23:12:49.080023 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.079985    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"]
Jul 06 23:12:49.081039 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.081009    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8" containerName="maas-api"
Jul 06 23:12:49.081039 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.081040    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8" containerName="maas-api"
Jul 06 23:12:49.081233 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.081140    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cdc07d6-73d8-4ce9-b337-82ab2aaa78a8" containerName="maas-api"
Jul 06 23:12:49.084359 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.084335    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.087008 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.086985    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c110c670-serving-cert\""
Jul 06 23:12:49.093491 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.093466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"]
Jul 06 23:12:49.204756 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.204712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b796d37-d45f-4d2f-8039-3baf071e1d81-openshift-service-ca\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.204756 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.204761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbgs5\" (UniqueName: \"kubernetes.io/projected/4b796d37-d45f-4d2f-8039-3baf071e1d81-kube-api-access-pbgs5\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.204990 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.204827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.305280 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.305247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.305464 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.305312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b796d37-d45f-4d2f-8039-3baf071e1d81-openshift-service-ca\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.305464 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.305341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbgs5\" (UniqueName: \"kubernetes.io/projected/4b796d37-d45f-4d2f-8039-3baf071e1d81-kube-api-access-pbgs5\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.305464 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:12:49.305423    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-serving-cert: secret "maas-api-e2e-ait-c110c670-serving-cert" not found
Jul 06 23:12:49.305677 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:12:49.305504    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls podName:4b796d37-d45f-4d2f-8039-3baf071e1d81 nodeName:}" failed. No retries permitted until 2026-07-06 23:12:49.805478259 +0000 UTC m=+1591.347809409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls") pod "maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2" (UID: "4b796d37-d45f-4d2f-8039-3baf071e1d81") : secret "maas-api-e2e-ait-c110c670-serving-cert" not found
Jul 06 23:12:49.306050 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.306031    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b796d37-d45f-4d2f-8039-3baf071e1d81-openshift-service-ca\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.319118 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.319083    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbgs5\" (UniqueName: \"kubernetes.io/projected/4b796d37-d45f-4d2f-8039-3baf071e1d81-kube-api-access-pbgs5\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.815732 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:49.815693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:49.815946 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:12:49.815862    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-serving-cert: secret "maas-api-e2e-ait-c110c670-serving-cert" not found
Jul 06 23:12:49.815946 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:12:49.815940    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls podName:4b796d37-d45f-4d2f-8039-3baf071e1d81 nodeName:}" failed. No retries permitted until 2026-07-06 23:12:50.815923152 +0000 UTC m=+1592.358254300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls") pod "maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2" (UID: "4b796d37-d45f-4d2f-8039-3baf071e1d81") : secret "maas-api-e2e-ait-c110c670-serving-cert" not found
Jul 06 23:12:50.825028 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:50.824983    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:50.827508 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:50.827485    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls\") pod \"maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:50.896920 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:50.896874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:51.037068 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:51.037036    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"]
Jul 06 23:12:51.040693 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:12:51.040662    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b796d37_d45f_4d2f_8039_3baf071e1d81.slice/crio-4c58a2c96cb64bef7be1bc52332d57013747b2b631e82ec97e8c0fe449fc0e77 WatchSource:0}: Error finding container 4c58a2c96cb64bef7be1bc52332d57013747b2b631e82ec97e8c0fe449fc0e77: Status 404 returned error can't find the container with id 4c58a2c96cb64bef7be1bc52332d57013747b2b631e82ec97e8c0fe449fc0e77
Jul 06 23:12:51.597954 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:51.597910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2" event={"ID":"4b796d37-d45f-4d2f-8039-3baf071e1d81","Type":"ContainerStarted","Data":"7c3291a3e01bf7659d971f1530c8a45bd28f8fd8aba527b79c64c0ce239d5d6b"}
Jul 06 23:12:51.598177 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:51.597975    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2" event={"ID":"4b796d37-d45f-4d2f-8039-3baf071e1d81","Type":"ContainerStarted","Data":"4c58a2c96cb64bef7be1bc52332d57013747b2b631e82ec97e8c0fe449fc0e77"}
Jul 06 23:12:51.598177 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:51.598009    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:51.613992 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:51.613924    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2" podStartSLOduration=2.238973426 podStartE2EDuration="2.613901473s" totalImagesPullingTime="374.928047ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:12:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:12:51.613759983 +0000 UTC m=+1593.156091155" watchObservedRunningTime="2026-07-06 23:12:51.613901473 +0000 UTC m=+1593.156232644"
Jul 06 23:12:54.373994 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.373953    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"]
Jul 06 23:12:54.374455 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.374247    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2" podUID="4b796d37-d45f-4d2f-8039-3baf071e1d81" containerName="maas-api" containerID="cri-o://7c3291a3e01bf7659d971f1530c8a45bd28f8fd8aba527b79c64c0ce239d5d6b" gracePeriod=30
Jul 06 23:12:54.612455 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.612418    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b796d37-d45f-4d2f-8039-3baf071e1d81" containerID="7c3291a3e01bf7659d971f1530c8a45bd28f8fd8aba527b79c64c0ce239d5d6b" exitCode=0
Jul 06 23:12:54.612577 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.612475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2" event={"ID":"4b796d37-d45f-4d2f-8039-3baf071e1d81","Type":"ContainerDied","Data":"7c3291a3e01bf7659d971f1530c8a45bd28f8fd8aba527b79c64c0ce239d5d6b"}
Jul 06 23:12:54.612577 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.612562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2" event={"ID":"4b796d37-d45f-4d2f-8039-3baf071e1d81","Type":"ContainerDied","Data":"4c58a2c96cb64bef7be1bc52332d57013747b2b631e82ec97e8c0fe449fc0e77"}
Jul 06 23:12:54.612695 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.612587    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c58a2c96cb64bef7be1bc52332d57013747b2b631e82ec97e8c0fe449fc0e77"
Jul 06 23:12:54.626335 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.626267    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:54.658170 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.658129    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbgs5\" (UniqueName: \"kubernetes.io/projected/4b796d37-d45f-4d2f-8039-3baf071e1d81-kube-api-access-pbgs5\") pod \"4b796d37-d45f-4d2f-8039-3baf071e1d81\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") "
Jul 06 23:12:54.658354 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.658244    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls\") pod \"4b796d37-d45f-4d2f-8039-3baf071e1d81\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") "
Jul 06 23:12:54.658354 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.658312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b796d37-d45f-4d2f-8039-3baf071e1d81-openshift-service-ca\") pod \"4b796d37-d45f-4d2f-8039-3baf071e1d81\" (UID: \"4b796d37-d45f-4d2f-8039-3baf071e1d81\") "
Jul 06 23:12:54.658696 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.658667    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b796d37-d45f-4d2f-8039-3baf071e1d81-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b796d37-d45f-4d2f-8039-3baf071e1d81" (UID: "4b796d37-d45f-4d2f-8039-3baf071e1d81"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:12:54.660346 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.660317    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b796d37-d45f-4d2f-8039-3baf071e1d81-kube-api-access-pbgs5" (OuterVolumeSpecName: "kube-api-access-pbgs5") pod "4b796d37-d45f-4d2f-8039-3baf071e1d81" (UID: "4b796d37-d45f-4d2f-8039-3baf071e1d81"). InnerVolumeSpecName "kube-api-access-pbgs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:12:54.660480 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.660465    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b796d37-d45f-4d2f-8039-3baf071e1d81" (UID: "4b796d37-d45f-4d2f-8039-3baf071e1d81"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:12:54.759958 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.759927    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b796d37-d45f-4d2f-8039-3baf071e1d81-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:12:54.759958 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.759957    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b796d37-d45f-4d2f-8039-3baf071e1d81-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:12:54.760177 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:54.759972    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbgs5\" (UniqueName: \"kubernetes.io/projected/4b796d37-d45f-4d2f-8039-3baf071e1d81-kube-api-access-pbgs5\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:12:55.617270 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:55.617240    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"
Jul 06 23:12:55.634103 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:55.634059    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"]
Jul 06 23:12:55.638357 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:55.638331    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c110c670-5b47884f4d-pj8b2"]
Jul 06 23:12:56.963321 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:12:56.963285    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b796d37-d45f-4d2f-8039-3baf071e1d81" path="/var/lib/kubelet/pods/4b796d37-d45f-4d2f-8039-3baf071e1d81/volumes"
Jul 06 23:13:07.467664 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.467583    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"]
Jul 06 23:13:07.468135 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.468114    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b796d37-d45f-4d2f-8039-3baf071e1d81" containerName="maas-api"
Jul 06 23:13:07.468178 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.468141    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b796d37-d45f-4d2f-8039-3baf071e1d81" containerName="maas-api"
Jul 06 23:13:07.468256 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.468245    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b796d37-d45f-4d2f-8039-3baf071e1d81" containerName="maas-api"
Jul 06 23:13:07.471953 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.471932    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:07.474679 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.474630    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d6f08872-serving-cert\""
Jul 06 23:13:07.484512 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.484486    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"]
Jul 06 23:13:07.568874 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.568843    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a945badf-196a-4996-b1d4-5f4eac055489-maas-api-tls\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:07.569047 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.568908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5xzt\" (UniqueName: \"kubernetes.io/projected/a945badf-196a-4996-b1d4-5f4eac055489-kube-api-access-m5xzt\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:07.569047 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.568987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a945badf-196a-4996-b1d4-5f4eac055489-openshift-service-ca\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:07.669756 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.669664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a945badf-196a-4996-b1d4-5f4eac055489-maas-api-tls\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:07.669756 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.669720    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5xzt\" (UniqueName: \"kubernetes.io/projected/a945badf-196a-4996-b1d4-5f4eac055489-kube-api-access-m5xzt\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:07.669756 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.669745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a945badf-196a-4996-b1d4-5f4eac055489-openshift-service-ca\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:07.670028 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:13:07.669812    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-serving-cert: secret "maas-api-e2e-derive-d6f08872-serving-cert" not found
Jul 06 23:13:07.670028 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:13:07.669877    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a945badf-196a-4996-b1d4-5f4eac055489-maas-api-tls podName:a945badf-196a-4996-b1d4-5f4eac055489 nodeName:}" failed. No retries permitted until 2026-07-06 23:13:08.169861866 +0000 UTC m=+1609.712193013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a945badf-196a-4996-b1d4-5f4eac055489-maas-api-tls") pod "maas-api-e2e-derive-d6f08872-f5cb45987-77gxn" (UID: "a945badf-196a-4996-b1d4-5f4eac055489") : secret "maas-api-e2e-derive-d6f08872-serving-cert" not found
Jul 06 23:13:07.670333 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.670316    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a945badf-196a-4996-b1d4-5f4eac055489-openshift-service-ca\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:07.681157 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:07.681129    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5xzt\" (UniqueName: \"kubernetes.io/projected/a945badf-196a-4996-b1d4-5f4eac055489-kube-api-access-m5xzt\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:08.174138 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:08.174097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a945badf-196a-4996-b1d4-5f4eac055489-maas-api-tls\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:08.176584 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:08.176563    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a945badf-196a-4996-b1d4-5f4eac055489-maas-api-tls\") pod \"maas-api-e2e-derive-d6f08872-f5cb45987-77gxn\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:08.385726 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:08.385689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:08.524215 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:08.524191    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"]
Jul 06 23:13:08.526916 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:13:08.526890    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda945badf_196a_4996_b1d4_5f4eac055489.slice/crio-1c8f07a6a24f297f433f5a0a696eae9553faa54d444ac7c727a8d9d36b7497e4 WatchSource:0}: Error finding container 1c8f07a6a24f297f433f5a0a696eae9553faa54d444ac7c727a8d9d36b7497e4: Status 404 returned error can't find the container with id 1c8f07a6a24f297f433f5a0a696eae9553faa54d444ac7c727a8d9d36b7497e4
Jul 06 23:13:08.675313 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:08.675274    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn" event={"ID":"a945badf-196a-4996-b1d4-5f4eac055489","Type":"ContainerStarted","Data":"1c8f07a6a24f297f433f5a0a696eae9553faa54d444ac7c727a8d9d36b7497e4"}
Jul 06 23:13:09.681796 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:09.681760    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn" event={"ID":"a945badf-196a-4996-b1d4-5f4eac055489","Type":"ContainerStarted","Data":"2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350"}
Jul 06 23:13:09.682191 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:09.681890    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:09.699783 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:09.699732    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn" podStartSLOduration=2.338071969 podStartE2EDuration="2.699716722s" totalImagesPullingTime="361.644753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:13:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:13:09.697066045 +0000 UTC m=+1611.239397206" watchObservedRunningTime="2026-07-06 23:13:09.699716722 +0000 UTC m=+1611.242047891"
Jul 06 23:13:12.203599 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.203560    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"]
Jul 06 23:13:12.204436 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.204398    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn" podUID="a945badf-196a-4996-b1d4-5f4eac055489" containerName="maas-api" containerID="cri-o://2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350" gracePeriod=30
Jul 06 23:13:12.458907 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.458839    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:12.523032 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.522991    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a945badf-196a-4996-b1d4-5f4eac055489-maas-api-tls\") pod \"a945badf-196a-4996-b1d4-5f4eac055489\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") "
Jul 06 23:13:12.523199 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.523057    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a945badf-196a-4996-b1d4-5f4eac055489-openshift-service-ca\") pod \"a945badf-196a-4996-b1d4-5f4eac055489\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") "
Jul 06 23:13:12.523199 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.523164    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5xzt\" (UniqueName: \"kubernetes.io/projected/a945badf-196a-4996-b1d4-5f4eac055489-kube-api-access-m5xzt\") pod \"a945badf-196a-4996-b1d4-5f4eac055489\" (UID: \"a945badf-196a-4996-b1d4-5f4eac055489\") "
Jul 06 23:13:12.523403 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.523378    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a945badf-196a-4996-b1d4-5f4eac055489-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a945badf-196a-4996-b1d4-5f4eac055489" (UID: "a945badf-196a-4996-b1d4-5f4eac055489"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:13:12.525257 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.525221    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a945badf-196a-4996-b1d4-5f4eac055489-kube-api-access-m5xzt" (OuterVolumeSpecName: "kube-api-access-m5xzt") pod "a945badf-196a-4996-b1d4-5f4eac055489" (UID: "a945badf-196a-4996-b1d4-5f4eac055489"). InnerVolumeSpecName "kube-api-access-m5xzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:13:12.525387 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.525319    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a945badf-196a-4996-b1d4-5f4eac055489-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a945badf-196a-4996-b1d4-5f4eac055489" (UID: "a945badf-196a-4996-b1d4-5f4eac055489"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:13:12.624179 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.624140    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5xzt\" (UniqueName: \"kubernetes.io/projected/a945badf-196a-4996-b1d4-5f4eac055489-kube-api-access-m5xzt\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:13:12.624179 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.624172    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a945badf-196a-4996-b1d4-5f4eac055489-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:13:12.624179 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.624183    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a945badf-196a-4996-b1d4-5f4eac055489-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:13:12.697197 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.697159    2563 generic.go:356] "Generic (PLEG): container finished" podID="a945badf-196a-4996-b1d4-5f4eac055489" containerID="2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350" exitCode=0
Jul 06 23:13:12.697405 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.697213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn" event={"ID":"a945badf-196a-4996-b1d4-5f4eac055489","Type":"ContainerDied","Data":"2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350"}
Jul 06 23:13:12.697405 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.697219    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"
Jul 06 23:13:12.697405 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.697303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn" event={"ID":"a945badf-196a-4996-b1d4-5f4eac055489","Type":"ContainerDied","Data":"1c8f07a6a24f297f433f5a0a696eae9553faa54d444ac7c727a8d9d36b7497e4"}
Jul 06 23:13:12.697405 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.697342    2563 scope.go:117] "RemoveContainer" containerID="2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350"
Jul 06 23:13:12.706074 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.706054    2563 scope.go:117] "RemoveContainer" containerID="2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350"
Jul 06 23:13:12.706361 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:13:12.706338    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350\": container with ID starting with 2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350 not found: ID does not exist" containerID="2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350"
Jul 06 23:13:12.706425 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.706371    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350"} err="failed to get container status \"2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350\": rpc error: code = NotFound desc = could not find container \"2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350\": container with ID starting with 2412fbec5d479363d0e31834772d7097fed36e3b29353e42029a9e6b61f4b350 not found: ID does not exist"
Jul 06 23:13:12.719579 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.719504    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"]
Jul 06 23:13:12.726257 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.726227    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d6f08872-f5cb45987-77gxn"]
Jul 06 23:13:12.964567 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:13:12.964534    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a945badf-196a-4996-b1d4-5f4eac055489" path="/var/lib/kubelet/pods/a945badf-196a-4996-b1d4-5f4eac055489/volumes"
Jul 06 23:14:55.666957 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.666834    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"]
Jul 06 23:14:55.669262 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.667411    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a945badf-196a-4996-b1d4-5f4eac055489" containerName="maas-api"
Jul 06 23:14:55.669262 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.667433    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a945badf-196a-4996-b1d4-5f4eac055489" containerName="maas-api"
Jul 06 23:14:55.669262 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.667541    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a945badf-196a-4996-b1d4-5f4eac055489" containerName="maas-api"
Jul 06 23:14:55.670321 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.670301    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.672837 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.672814    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bd8ffd94-serving-cert\""
Jul 06 23:14:55.682154 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.682127    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"]
Jul 06 23:14:55.726528 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.726498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f037a50-313e-4e48-ae7a-4262b1a89071-maas-api-tls\") pod \"maas-api-e2e-mt-bd8ffd94-748c678c59-75962\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.726528 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.726533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfwsz\" (UniqueName: \"kubernetes.io/projected/6f037a50-313e-4e48-ae7a-4262b1a89071-kube-api-access-kfwsz\") pod \"maas-api-e2e-mt-bd8ffd94-748c678c59-75962\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.726775 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.726623    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f037a50-313e-4e48-ae7a-4262b1a89071-openshift-service-ca\") pod \"maas-api-e2e-mt-bd8ffd94-748c678c59-75962\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.828015 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.827982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f037a50-313e-4e48-ae7a-4262b1a89071-maas-api-tls\") pod \"maas-api-e2e-mt-bd8ffd94-748c678c59-75962\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.828015 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.828025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfwsz\" (UniqueName: \"kubernetes.io/projected/6f037a50-313e-4e48-ae7a-4262b1a89071-kube-api-access-kfwsz\") pod \"maas-api-e2e-mt-bd8ffd94-748c678c59-75962\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.828277 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.828078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f037a50-313e-4e48-ae7a-4262b1a89071-openshift-service-ca\") pod \"maas-api-e2e-mt-bd8ffd94-748c678c59-75962\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.828727 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.828708    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f037a50-313e-4e48-ae7a-4262b1a89071-openshift-service-ca\") pod \"maas-api-e2e-mt-bd8ffd94-748c678c59-75962\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.830598 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.830575    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f037a50-313e-4e48-ae7a-4262b1a89071-maas-api-tls\") pod \"maas-api-e2e-mt-bd8ffd94-748c678c59-75962\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.837001 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.836974    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfwsz\" (UniqueName: \"kubernetes.io/projected/6f037a50-313e-4e48-ae7a-4262b1a89071-kube-api-access-kfwsz\") pod \"maas-api-e2e-mt-bd8ffd94-748c678c59-75962\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:55.981923 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:55.981882    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:56.129226 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:56.129184    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"]
Jul 06 23:14:56.129405 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:14:56.129287    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f037a50_313e_4e48_ae7a_4262b1a89071.slice/crio-f226133973dee68790d28fcc59afb8b75590ec0e0fcb92018d39737942b7f03d WatchSource:0}: Error finding container f226133973dee68790d28fcc59afb8b75590ec0e0fcb92018d39737942b7f03d: Status 404 returned error can't find the container with id f226133973dee68790d28fcc59afb8b75590ec0e0fcb92018d39737942b7f03d
Jul 06 23:14:56.148838 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:56.148802    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962" event={"ID":"6f037a50-313e-4e48-ae7a-4262b1a89071","Type":"ContainerStarted","Data":"f226133973dee68790d28fcc59afb8b75590ec0e0fcb92018d39737942b7f03d"}
Jul 06 23:14:57.155193 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:57.155154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962" event={"ID":"6f037a50-313e-4e48-ae7a-4262b1a89071","Type":"ContainerStarted","Data":"63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b"}
Jul 06 23:14:57.155613 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:57.155252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:14:57.173527 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:14:57.173472    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962" podStartSLOduration=1.7332694530000001 podStartE2EDuration="2.173456164s" totalImagesPullingTime="440.186711ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:14:57.171712028 +0000 UTC m=+1718.714043200" watchObservedRunningTime="2026-07-06 23:14:57.173456164 +0000 UTC m=+1718.715787333"
Jul 06 23:15:00.137316 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:00.137275    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"]
Jul 06 23:15:00.140760 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:00.140739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"
Jul 06 23:15:00.148681 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:00.148632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"]
Jul 06 23:15:00.267934 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:00.267894    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7zkq\" (UniqueName: \"kubernetes.io/projected/10fe2105-f12d-46b9-947d-5975a03b52ef-kube-api-access-j7zkq\") pod \"maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9\" (UID: \"10fe2105-f12d-46b9-947d-5975a03b52ef\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"
Jul 06 23:15:00.369006 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:00.368972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7zkq\" (UniqueName: \"kubernetes.io/projected/10fe2105-f12d-46b9-947d-5975a03b52ef-kube-api-access-j7zkq\") pod \"maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9\" (UID: \"10fe2105-f12d-46b9-947d-5975a03b52ef\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"
Jul 06 23:15:00.377056 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:00.377024    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7zkq\" (UniqueName: \"kubernetes.io/projected/10fe2105-f12d-46b9-947d-5975a03b52ef-kube-api-access-j7zkq\") pod \"maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9\" (UID: \"10fe2105-f12d-46b9-947d-5975a03b52ef\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"
Jul 06 23:15:00.452878 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:00.452842    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"
Jul 06 23:15:00.586164 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:00.586010    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"]
Jul 06 23:15:00.588519 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:15:00.588487    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10fe2105_f12d_46b9_947d_5975a03b52ef.slice/crio-e1638f7e654ae09073295d2afc1490bd7887444692eef0efc587fc742547c8db WatchSource:0}: Error finding container e1638f7e654ae09073295d2afc1490bd7887444692eef0efc587fc742547c8db: Status 404 returned error can't find the container with id e1638f7e654ae09073295d2afc1490bd7887444692eef0efc587fc742547c8db
Jul 06 23:15:01.172730 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:01.172691    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9" event={"ID":"10fe2105-f12d-46b9-947d-5975a03b52ef","Type":"ContainerStarted","Data":"e1638f7e654ae09073295d2afc1490bd7887444692eef0efc587fc742547c8db"}
Jul 06 23:15:03.164824 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:03.164743    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:15:03.184034 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:03.184000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9" event={"ID":"10fe2105-f12d-46b9-947d-5975a03b52ef","Type":"ContainerStarted","Data":"e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9"}
Jul 06 23:15:03.202279 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:03.202222    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9" podStartSLOduration=1.028438061 podStartE2EDuration="3.202203253s" totalImagesPullingTime="2.173765192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:15:03.200403879 +0000 UTC m=+1724.742735049" watchObservedRunningTime="2026-07-06 23:15:03.202203253 +0000 UTC m=+1724.744534423"
Jul 06 23:15:07.908176 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:07.908131    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"]
Jul 06 23:15:07.908810 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:07.908383    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962" podUID="6f037a50-313e-4e48-ae7a-4262b1a89071" containerName="maas-api" containerID="cri-o://63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b" gracePeriod=30
Jul 06 23:15:07.943294 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:07.943260    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"]
Jul 06 23:15:07.943466 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:07.943427    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9" podUID="10fe2105-f12d-46b9-947d-5975a03b52ef" containerName="cleanup" containerID="cri-o://e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9" gracePeriod=30
Jul 06 23:15:08.157489 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.157463    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:15:08.207880 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.207847    2563 generic.go:356] "Generic (PLEG): container finished" podID="6f037a50-313e-4e48-ae7a-4262b1a89071" containerID="63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b" exitCode=0
Jul 06 23:15:08.208063 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.207892    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"
Jul 06 23:15:08.208063 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.207892    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962" event={"ID":"6f037a50-313e-4e48-ae7a-4262b1a89071","Type":"ContainerDied","Data":"63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b"}
Jul 06 23:15:08.208063 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.207961    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962" event={"ID":"6f037a50-313e-4e48-ae7a-4262b1a89071","Type":"ContainerDied","Data":"f226133973dee68790d28fcc59afb8b75590ec0e0fcb92018d39737942b7f03d"}
Jul 06 23:15:08.208063 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.207991    2563 scope.go:117] "RemoveContainer" containerID="63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b"
Jul 06 23:15:08.216717 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.216698    2563 scope.go:117] "RemoveContainer" containerID="63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b"
Jul 06 23:15:08.216992 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:15:08.216964    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b\": container with ID starting with 63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b not found: ID does not exist" containerID="63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b"
Jul 06 23:15:08.217049 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.216996    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b"} err="failed to get container status \"63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b\": rpc error: code = NotFound desc = could not find container \"63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b\": container with ID starting with 63b1fab85d25526ca0ea10f2124d295e5ce38ab44907303b93cee124608b489b not found: ID does not exist"
Jul 06 23:15:08.238449 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.238409    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f037a50-313e-4e48-ae7a-4262b1a89071-openshift-service-ca\") pod \"6f037a50-313e-4e48-ae7a-4262b1a89071\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") "
Jul 06 23:15:08.238617 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.238471    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfwsz\" (UniqueName: \"kubernetes.io/projected/6f037a50-313e-4e48-ae7a-4262b1a89071-kube-api-access-kfwsz\") pod \"6f037a50-313e-4e48-ae7a-4262b1a89071\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") "
Jul 06 23:15:08.238617 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.238521    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f037a50-313e-4e48-ae7a-4262b1a89071-maas-api-tls\") pod \"6f037a50-313e-4e48-ae7a-4262b1a89071\" (UID: \"6f037a50-313e-4e48-ae7a-4262b1a89071\") "
Jul 06 23:15:08.238888 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.238856    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f037a50-313e-4e48-ae7a-4262b1a89071-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f037a50-313e-4e48-ae7a-4262b1a89071" (UID: "6f037a50-313e-4e48-ae7a-4262b1a89071"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:15:08.241187 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.241147    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f037a50-313e-4e48-ae7a-4262b1a89071-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f037a50-313e-4e48-ae7a-4262b1a89071" (UID: "6f037a50-313e-4e48-ae7a-4262b1a89071"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:15:08.241347 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.241196    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f037a50-313e-4e48-ae7a-4262b1a89071-kube-api-access-kfwsz" (OuterVolumeSpecName: "kube-api-access-kfwsz") pod "6f037a50-313e-4e48-ae7a-4262b1a89071" (UID: "6f037a50-313e-4e48-ae7a-4262b1a89071"). InnerVolumeSpecName "kube-api-access-kfwsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:15:08.340059 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.340003    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f037a50-313e-4e48-ae7a-4262b1a89071-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:15:08.340059 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.340052    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfwsz\" (UniqueName: \"kubernetes.io/projected/6f037a50-313e-4e48-ae7a-4262b1a89071-kube-api-access-kfwsz\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:15:08.340059 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.340062    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f037a50-313e-4e48-ae7a-4262b1a89071-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:15:08.530995 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.530947    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"]
Jul 06 23:15:08.535564 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.535503    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd8ffd94-748c678c59-75962"]
Jul 06 23:15:08.962804 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:08.962772    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f037a50-313e-4e48-ae7a-4262b1a89071" path="/var/lib/kubelet/pods/6f037a50-313e-4e48-ae7a-4262b1a89071/volumes"
Jul 06 23:15:37.005455 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.005425    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"]
Jul 06 23:15:37.005955 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.005907    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f037a50-313e-4e48-ae7a-4262b1a89071" containerName="maas-api"
Jul 06 23:15:37.005955 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.005925    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f037a50-313e-4e48-ae7a-4262b1a89071" containerName="maas-api"
Jul 06 23:15:37.006064 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.006026    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f037a50-313e-4e48-ae7a-4262b1a89071" containerName="maas-api"
Jul 06 23:15:37.011253 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.011232    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.013629 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.013609    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2fa3353e-serving-cert\""
Jul 06 23:15:37.021430 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.021403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"]
Jul 06 23:15:37.074846 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.074803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1408e0e0-9ea1-4d19-8941-9825e83d7814-openshift-service-ca\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.075019 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.074860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1408e0e0-9ea1-4d19-8941-9825e83d7814-maas-api-tls\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.075019 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.074910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgdrm\" (UniqueName: \"kubernetes.io/projected/1408e0e0-9ea1-4d19-8941-9825e83d7814-kube-api-access-dgdrm\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.176055 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.176018    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1408e0e0-9ea1-4d19-8941-9825e83d7814-openshift-service-ca\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.176231 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.176061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1408e0e0-9ea1-4d19-8941-9825e83d7814-maas-api-tls\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.176231 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.176098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgdrm\" (UniqueName: \"kubernetes.io/projected/1408e0e0-9ea1-4d19-8941-9825e83d7814-kube-api-access-dgdrm\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.176231 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:15:37.176194    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-serving-cert: secret "maas-api-e2e-mt-2fa3353e-serving-cert" not found
Jul 06 23:15:37.176336 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:15:37.176287    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1408e0e0-9ea1-4d19-8941-9825e83d7814-maas-api-tls podName:1408e0e0-9ea1-4d19-8941-9825e83d7814 nodeName:}" failed. No retries permitted until 2026-07-06 23:15:37.676263248 +0000 UTC m=+1759.218594400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1408e0e0-9ea1-4d19-8941-9825e83d7814-maas-api-tls") pod "maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l" (UID: "1408e0e0-9ea1-4d19-8941-9825e83d7814") : secret "maas-api-e2e-mt-2fa3353e-serving-cert" not found
Jul 06 23:15:37.176835 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.176816    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1408e0e0-9ea1-4d19-8941-9825e83d7814-openshift-service-ca\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.186050 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.186019    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgdrm\" (UniqueName: \"kubernetes.io/projected/1408e0e0-9ea1-4d19-8941-9825e83d7814-kube-api-access-dgdrm\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.679564 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.679524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1408e0e0-9ea1-4d19-8941-9825e83d7814-maas-api-tls\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.682030 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.682000    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1408e0e0-9ea1-4d19-8941-9825e83d7814-maas-api-tls\") pod \"maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:37.923156 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:37.923115    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:38.073337 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.073302    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"]
Jul 06 23:15:38.076791 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:15:38.076747    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1408e0e0_9ea1_4d19_8941_9825e83d7814.slice/crio-b5d83edda92417a97e38500cbe98ef5a282132b408c58504ba4ad6f0e4569056 WatchSource:0}: Error finding container b5d83edda92417a97e38500cbe98ef5a282132b408c58504ba4ad6f0e4569056: Status 404 returned error can't find the container with id b5d83edda92417a97e38500cbe98ef5a282132b408c58504ba4ad6f0e4569056
Jul 06 23:15:38.129400 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.129366    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"
Jul 06 23:15:38.183916 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.183795    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7zkq\" (UniqueName: \"kubernetes.io/projected/10fe2105-f12d-46b9-947d-5975a03b52ef-kube-api-access-j7zkq\") pod \"10fe2105-f12d-46b9-947d-5975a03b52ef\" (UID: \"10fe2105-f12d-46b9-947d-5975a03b52ef\") "
Jul 06 23:15:38.186115 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.186078    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10fe2105-f12d-46b9-947d-5975a03b52ef-kube-api-access-j7zkq" (OuterVolumeSpecName: "kube-api-access-j7zkq") pod "10fe2105-f12d-46b9-947d-5975a03b52ef" (UID: "10fe2105-f12d-46b9-947d-5975a03b52ef"). InnerVolumeSpecName "kube-api-access-j7zkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:15:38.285014 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.284978    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7zkq\" (UniqueName: \"kubernetes.io/projected/10fe2105-f12d-46b9-947d-5975a03b52ef-kube-api-access-j7zkq\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:15:38.347775 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.347739    2563 generic.go:356] "Generic (PLEG): container finished" podID="10fe2105-f12d-46b9-947d-5975a03b52ef" containerID="e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9" exitCode=137
Jul 06 23:15:38.347985 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.347818    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"
Jul 06 23:15:38.347985 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.347841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9" event={"ID":"10fe2105-f12d-46b9-947d-5975a03b52ef","Type":"ContainerDied","Data":"e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9"}
Jul 06 23:15:38.347985 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.347923    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9" event={"ID":"10fe2105-f12d-46b9-947d-5975a03b52ef","Type":"ContainerDied","Data":"e1638f7e654ae09073295d2afc1490bd7887444692eef0efc587fc742547c8db"}
Jul 06 23:15:38.347985 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.347946    2563 scope.go:117] "RemoveContainer" containerID="e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9"
Jul 06 23:15:38.349075 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.349051    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l" event={"ID":"1408e0e0-9ea1-4d19-8941-9825e83d7814","Type":"ContainerStarted","Data":"b5d83edda92417a97e38500cbe98ef5a282132b408c58504ba4ad6f0e4569056"}
Jul 06 23:15:38.356743 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.356720    2563 scope.go:117] "RemoveContainer" containerID="e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9"
Jul 06 23:15:38.357044 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:15:38.357021    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9\": container with ID starting with e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9 not found: ID does not exist" containerID="e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9"
Jul 06 23:15:38.357096 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.357054    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9"} err="failed to get container status \"e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9\": rpc error: code = NotFound desc = could not find container \"e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9\": container with ID starting with e437f726788232928235359e359da1f27a426d722515ddfecc04bad5f4c546b9 not found: ID does not exist"
Jul 06 23:15:38.368489 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.368452    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"]
Jul 06 23:15:38.375609 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.375583    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-bd8ffd94-29722995-bjbl9"]
Jul 06 23:15:38.963240 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:38.963204    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10fe2105-f12d-46b9-947d-5975a03b52ef" path="/var/lib/kubelet/pods/10fe2105-f12d-46b9-947d-5975a03b52ef/volumes"
Jul 06 23:15:39.355447 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:39.355361    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l" event={"ID":"1408e0e0-9ea1-4d19-8941-9825e83d7814","Type":"ContainerStarted","Data":"0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb"}
Jul 06 23:15:39.355836 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:39.355499    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:15:39.372611 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:39.372565    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l" podStartSLOduration=2.972580245 podStartE2EDuration="3.372547666s" totalImagesPullingTime="399.967421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:15:39.371218045 +0000 UTC m=+1760.913549217" watchObservedRunningTime="2026-07-06 23:15:39.372547666 +0000 UTC m=+1760.914878835"
Jul 06 23:15:45.365081 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:15:45.365050    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:16:14.032281 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.032181    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"]
Jul 06 23:16:14.033075 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.033035    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l" podUID="1408e0e0-9ea1-4d19-8941-9825e83d7814" containerName="maas-api" containerID="cri-o://0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb" gracePeriod=30
Jul 06 23:16:14.277018 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.276991    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:16:14.412053 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.411958    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1408e0e0-9ea1-4d19-8941-9825e83d7814-maas-api-tls\") pod \"1408e0e0-9ea1-4d19-8941-9825e83d7814\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") "
Jul 06 23:16:14.412053 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.411993    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgdrm\" (UniqueName: \"kubernetes.io/projected/1408e0e0-9ea1-4d19-8941-9825e83d7814-kube-api-access-dgdrm\") pod \"1408e0e0-9ea1-4d19-8941-9825e83d7814\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") "
Jul 06 23:16:14.412053 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.412013    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1408e0e0-9ea1-4d19-8941-9825e83d7814-openshift-service-ca\") pod \"1408e0e0-9ea1-4d19-8941-9825e83d7814\" (UID: \"1408e0e0-9ea1-4d19-8941-9825e83d7814\") "
Jul 06 23:16:14.412441 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.412415    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1408e0e0-9ea1-4d19-8941-9825e83d7814-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1408e0e0-9ea1-4d19-8941-9825e83d7814" (UID: "1408e0e0-9ea1-4d19-8941-9825e83d7814"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:16:14.414291 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.414256    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1408e0e0-9ea1-4d19-8941-9825e83d7814-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1408e0e0-9ea1-4d19-8941-9825e83d7814" (UID: "1408e0e0-9ea1-4d19-8941-9825e83d7814"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:16:14.414409 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.414295    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1408e0e0-9ea1-4d19-8941-9825e83d7814-kube-api-access-dgdrm" (OuterVolumeSpecName: "kube-api-access-dgdrm") pod "1408e0e0-9ea1-4d19-8941-9825e83d7814" (UID: "1408e0e0-9ea1-4d19-8941-9825e83d7814"). InnerVolumeSpecName "kube-api-access-dgdrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:16:14.513043 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.513012    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1408e0e0-9ea1-4d19-8941-9825e83d7814-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:16:14.513043 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.513037    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgdrm\" (UniqueName: \"kubernetes.io/projected/1408e0e0-9ea1-4d19-8941-9825e83d7814-kube-api-access-dgdrm\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:16:14.513249 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.513054    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1408e0e0-9ea1-4d19-8941-9825e83d7814-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:16:14.517714 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.517681    2563 generic.go:356] "Generic (PLEG): container finished" podID="1408e0e0-9ea1-4d19-8941-9825e83d7814" containerID="0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb" exitCode=0
Jul 06 23:16:14.517841 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.517735    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"
Jul 06 23:16:14.517841 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.517731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l" event={"ID":"1408e0e0-9ea1-4d19-8941-9825e83d7814","Type":"ContainerDied","Data":"0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb"}
Jul 06 23:16:14.517909 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.517872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l" event={"ID":"1408e0e0-9ea1-4d19-8941-9825e83d7814","Type":"ContainerDied","Data":"b5d83edda92417a97e38500cbe98ef5a282132b408c58504ba4ad6f0e4569056"}
Jul 06 23:16:14.517909 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.517904    2563 scope.go:117] "RemoveContainer" containerID="0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb"
Jul 06 23:16:14.526631 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.526615    2563 scope.go:117] "RemoveContainer" containerID="0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb"
Jul 06 23:16:14.526967 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:16:14.526951    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb\": container with ID starting with 0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb not found: ID does not exist" containerID="0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb"
Jul 06 23:16:14.527011 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.526977    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb"} err="failed to get container status \"0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb\": rpc error: code = NotFound desc = could not find container \"0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb\": container with ID starting with 0212af9b82725495d5dcfb17b57dd590ba003dcd62398192fface9c8f2fda9eb not found: ID does not exist"
Jul 06 23:16:14.538783 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.538757    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"]
Jul 06 23:16:14.542517 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.542497    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2fa3353e-78dfc4c494-4v22l"]
Jul 06 23:16:14.962826 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:16:14.962795    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1408e0e0-9ea1-4d19-8941-9825e83d7814" path="/var/lib/kubelet/pods/1408e0e0-9ea1-4d19-8941-9825e83d7814/volumes"
Jul 06 23:18:27.881882 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.881849    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"]
Jul 06 23:18:27.882295 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.882160    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10fe2105-f12d-46b9-947d-5975a03b52ef" containerName="cleanup"
Jul 06 23:18:27.882295 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.882172    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10fe2105-f12d-46b9-947d-5975a03b52ef" containerName="cleanup"
Jul 06 23:18:27.882295 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.882190    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1408e0e0-9ea1-4d19-8941-9825e83d7814" containerName="maas-api"
Jul 06 23:18:27.882295 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.882196    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1408e0e0-9ea1-4d19-8941-9825e83d7814" containerName="maas-api"
Jul 06 23:18:27.882295 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.882259    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1408e0e0-9ea1-4d19-8941-9825e83d7814" containerName="maas-api"
Jul 06 23:18:27.882295 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.882267    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="10fe2105-f12d-46b9-947d-5975a03b52ef" containerName="cleanup"
Jul 06 23:18:27.885157 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.885141    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:27.887619 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.887594    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2ff615-serving-cert\""
Jul 06 23:18:27.895187 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.895161    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"]
Jul 06 23:18:27.951192 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.951153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbnxh\" (UniqueName: \"kubernetes.io/projected/83230b6b-9338-4887-b60a-c916ff4410be-kube-api-access-fbnxh\") pod \"maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:27.951381 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.951204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83230b6b-9338-4887-b60a-c916ff4410be-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:27.951381 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:27.951231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83230b6b-9338-4887-b60a-c916ff4410be-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:28.052452 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:28.052408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbnxh\" (UniqueName: \"kubernetes.io/projected/83230b6b-9338-4887-b60a-c916ff4410be-kube-api-access-fbnxh\") pod \"maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:28.052708 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:28.052481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83230b6b-9338-4887-b60a-c916ff4410be-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:28.052708 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:28.052505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83230b6b-9338-4887-b60a-c916ff4410be-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:28.053224 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:28.053201    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83230b6b-9338-4887-b60a-c916ff4410be-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:28.055053 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:28.055027    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83230b6b-9338-4887-b60a-c916ff4410be-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:28.062101 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:28.062074    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbnxh\" (UniqueName: \"kubernetes.io/projected/83230b6b-9338-4887-b60a-c916ff4410be-kube-api-access-fbnxh\") pod \"maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:28.196891 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:28.196850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:28.341625 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:28.341599    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"]
Jul 06 23:18:28.344025 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:18:28.343998    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83230b6b_9338_4887_b60a_c916ff4410be.slice/crio-181983abe691cba1d7a656c40fd9826d507828017d896dbab41e6878aeccd074 WatchSource:0}: Error finding container 181983abe691cba1d7a656c40fd9826d507828017d896dbab41e6878aeccd074: Status 404 returned error can't find the container with id 181983abe691cba1d7a656c40fd9826d507828017d896dbab41e6878aeccd074
Jul 06 23:18:28.345351 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:28.345319    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 23:18:29.093364 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:29.093251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp" event={"ID":"83230b6b-9338-4887-b60a-c916ff4410be","Type":"ContainerStarted","Data":"a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888"}
Jul 06 23:18:29.093364 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:29.093308    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp" event={"ID":"83230b6b-9338-4887-b60a-c916ff4410be","Type":"ContainerStarted","Data":"181983abe691cba1d7a656c40fd9826d507828017d896dbab41e6878aeccd074"}
Jul 06 23:18:29.093882 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:29.093383    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:29.113482 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:29.113425    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp" podStartSLOduration=1.6403895149999999 podStartE2EDuration="2.113407716s" totalImagesPullingTime="473.018201ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:18:29.11191202 +0000 UTC m=+1930.654243190" watchObservedRunningTime="2026-07-06 23:18:29.113407716 +0000 UTC m=+1930.655738885"
Jul 06 23:18:35.104669 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:35.104622    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:18:35.943449 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:35.943401    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"]
Jul 06 23:18:35.947345 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:35.947311    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:35.950185 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:35.950144    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f9d5b5-serving-cert\""
Jul 06 23:18:35.957482 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:35.957452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"]
Jul 06 23:18:36.024230 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.024190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9x8r\" (UniqueName: \"kubernetes.io/projected/a98bb539-20d1-47bb-a093-b61c84d58289-kube-api-access-s9x8r\") pod \"maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.024230 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.024231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98bb539-20d1-47bb-a093-b61c84d58289-maas-api-tls\") pod \"maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.024475 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.024258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98bb539-20d1-47bb-a093-b61c84d58289-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.125383 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.125319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9x8r\" (UniqueName: \"kubernetes.io/projected/a98bb539-20d1-47bb-a093-b61c84d58289-kube-api-access-s9x8r\") pod \"maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.125854 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.125391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98bb539-20d1-47bb-a093-b61c84d58289-maas-api-tls\") pod \"maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.125854 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.125443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98bb539-20d1-47bb-a093-b61c84d58289-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.126302 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.126271    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98bb539-20d1-47bb-a093-b61c84d58289-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.128264 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.128240    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98bb539-20d1-47bb-a093-b61c84d58289-maas-api-tls\") pod \"maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.134674 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.134615    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9x8r\" (UniqueName: \"kubernetes.io/projected/a98bb539-20d1-47bb-a093-b61c84d58289-kube-api-access-s9x8r\") pod \"maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.262546 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.262458    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:36.414608 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:36.414556    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"]
Jul 06 23:18:36.420255 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:18:36.420208    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98bb539_20d1_47bb_a093_b61c84d58289.slice/crio-efb18a21f0efdeb6b7ed14300435f8d538b2e46029946a34d87eb993f9ce5f7d WatchSource:0}: Error finding container efb18a21f0efdeb6b7ed14300435f8d538b2e46029946a34d87eb993f9ce5f7d: Status 404 returned error can't find the container with id efb18a21f0efdeb6b7ed14300435f8d538b2e46029946a34d87eb993f9ce5f7d
Jul 06 23:18:37.134998 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:37.134891    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t" event={"ID":"a98bb539-20d1-47bb-a093-b61c84d58289","Type":"ContainerStarted","Data":"d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f"}
Jul 06 23:18:37.134998 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:37.134944    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t" event={"ID":"a98bb539-20d1-47bb-a093-b61c84d58289","Type":"ContainerStarted","Data":"efb18a21f0efdeb6b7ed14300435f8d538b2e46029946a34d87eb993f9ce5f7d"}
Jul 06 23:18:37.134998 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:37.134977    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:18:37.150987 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:37.150937    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t" podStartSLOduration=1.828390583 podStartE2EDuration="2.150920931s" totalImagesPullingTime="322.530348ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:18:37.150526012 +0000 UTC m=+1938.692857189" watchObservedRunningTime="2026-07-06 23:18:37.150920931 +0000 UTC m=+1938.693252100"
Jul 06 23:18:43.146197 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:18:43.146166    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:19:19.076999 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:19.076920    2563 scope.go:117] "RemoveContainer" containerID="7c3291a3e01bf7659d971f1530c8a45bd28f8fd8aba527b79c64c0ce239d5d6b"
Jul 06 23:19:45.790699 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:45.790655    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"]
Jul 06 23:19:45.791164 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:45.790878    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp" podUID="83230b6b-9338-4887-b60a-c916ff4410be" containerName="maas-api" containerID="cri-o://a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888" gracePeriod=30
Jul 06 23:19:46.232964 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.232941    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:19:46.330339 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.330241    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83230b6b-9338-4887-b60a-c916ff4410be-maas-api-tls\") pod \"83230b6b-9338-4887-b60a-c916ff4410be\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") "
Jul 06 23:19:46.330522 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.330360    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83230b6b-9338-4887-b60a-c916ff4410be-openshift-service-ca\") pod \"83230b6b-9338-4887-b60a-c916ff4410be\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") "
Jul 06 23:19:46.330522 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.330427    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbnxh\" (UniqueName: \"kubernetes.io/projected/83230b6b-9338-4887-b60a-c916ff4410be-kube-api-access-fbnxh\") pod \"83230b6b-9338-4887-b60a-c916ff4410be\" (UID: \"83230b6b-9338-4887-b60a-c916ff4410be\") "
Jul 06 23:19:46.330745 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.330715    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83230b6b-9338-4887-b60a-c916ff4410be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83230b6b-9338-4887-b60a-c916ff4410be" (UID: "83230b6b-9338-4887-b60a-c916ff4410be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:19:46.332683 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.332617    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83230b6b-9338-4887-b60a-c916ff4410be-kube-api-access-fbnxh" (OuterVolumeSpecName: "kube-api-access-fbnxh") pod "83230b6b-9338-4887-b60a-c916ff4410be" (UID: "83230b6b-9338-4887-b60a-c916ff4410be"). InnerVolumeSpecName "kube-api-access-fbnxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:19:46.332683 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.332628    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83230b6b-9338-4887-b60a-c916ff4410be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83230b6b-9338-4887-b60a-c916ff4410be" (UID: "83230b6b-9338-4887-b60a-c916ff4410be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:19:46.431565 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.431520    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83230b6b-9338-4887-b60a-c916ff4410be-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:19:46.431565 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.431556    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83230b6b-9338-4887-b60a-c916ff4410be-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:19:46.431565 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.431570    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbnxh\" (UniqueName: \"kubernetes.io/projected/83230b6b-9338-4887-b60a-c916ff4410be-kube-api-access-fbnxh\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:19:46.451388 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.451347    2563 generic.go:356] "Generic (PLEG): container finished" podID="83230b6b-9338-4887-b60a-c916ff4410be" containerID="a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888" exitCode=0
Jul 06 23:19:46.451388 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.451391    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"
Jul 06 23:19:46.451574 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.451434    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp" event={"ID":"83230b6b-9338-4887-b60a-c916ff4410be","Type":"ContainerDied","Data":"a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888"}
Jul 06 23:19:46.451574 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.451485    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp" event={"ID":"83230b6b-9338-4887-b60a-c916ff4410be","Type":"ContainerDied","Data":"181983abe691cba1d7a656c40fd9826d507828017d896dbab41e6878aeccd074"}
Jul 06 23:19:46.451574 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.451507    2563 scope.go:117] "RemoveContainer" containerID="a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888"
Jul 06 23:19:46.460556 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.460541    2563 scope.go:117] "RemoveContainer" containerID="a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888"
Jul 06 23:19:46.460872 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:19:46.460850    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888\": container with ID starting with a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888 not found: ID does not exist" containerID="a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888"
Jul 06 23:19:46.460950 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.460883    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888"} err="failed to get container status \"a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888\": rpc error: code = NotFound desc = could not find container \"a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888\": container with ID starting with a63153b6a470f3149850258d0b55d8a4f80f77b86455b971c4750e6931c1a888 not found: ID does not exist"
Jul 06 23:19:46.472131 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.472104    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"]
Jul 06 23:19:46.475443 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.475421    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ff615-6d56476cd8-82fqp"]
Jul 06 23:19:46.962773 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:19:46.962741    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83230b6b-9338-4887-b60a-c916ff4410be" path="/var/lib/kubelet/pods/83230b6b-9338-4887-b60a-c916ff4410be/volumes"
Jul 06 23:20:28.827374 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:28.827331    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"]
Jul 06 23:20:28.828174 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:28.828122    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t" podUID="a98bb539-20d1-47bb-a093-b61c84d58289" containerName="maas-api" containerID="cri-o://d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f" gracePeriod=30
Jul 06 23:20:29.077674 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.077570    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:20:29.198520 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.198478    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98bb539-20d1-47bb-a093-b61c84d58289-openshift-service-ca\") pod \"a98bb539-20d1-47bb-a093-b61c84d58289\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") "
Jul 06 23:20:29.198723 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.198558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9x8r\" (UniqueName: \"kubernetes.io/projected/a98bb539-20d1-47bb-a093-b61c84d58289-kube-api-access-s9x8r\") pod \"a98bb539-20d1-47bb-a093-b61c84d58289\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") "
Jul 06 23:20:29.198723 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.198605    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98bb539-20d1-47bb-a093-b61c84d58289-maas-api-tls\") pod \"a98bb539-20d1-47bb-a093-b61c84d58289\" (UID: \"a98bb539-20d1-47bb-a093-b61c84d58289\") "
Jul 06 23:20:29.198955 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.198924    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a98bb539-20d1-47bb-a093-b61c84d58289-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a98bb539-20d1-47bb-a093-b61c84d58289" (UID: "a98bb539-20d1-47bb-a093-b61c84d58289"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:20:29.200926 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.200899    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a98bb539-20d1-47bb-a093-b61c84d58289-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a98bb539-20d1-47bb-a093-b61c84d58289" (UID: "a98bb539-20d1-47bb-a093-b61c84d58289"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:20:29.201001 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.200943    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98bb539-20d1-47bb-a093-b61c84d58289-kube-api-access-s9x8r" (OuterVolumeSpecName: "kube-api-access-s9x8r") pod "a98bb539-20d1-47bb-a093-b61c84d58289" (UID: "a98bb539-20d1-47bb-a093-b61c84d58289"). InnerVolumeSpecName "kube-api-access-s9x8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:20:29.299621 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.299573    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9x8r\" (UniqueName: \"kubernetes.io/projected/a98bb539-20d1-47bb-a093-b61c84d58289-kube-api-access-s9x8r\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:20:29.299621 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.299616    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a98bb539-20d1-47bb-a093-b61c84d58289-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:20:29.299621 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.299627    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a98bb539-20d1-47bb-a093-b61c84d58289-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:20:29.637508 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.637461    2563 generic.go:356] "Generic (PLEG): container finished" podID="a98bb539-20d1-47bb-a093-b61c84d58289" containerID="d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f" exitCode=0
Jul 06 23:20:29.637508 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.637511    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"
Jul 06 23:20:29.637788 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.637525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t" event={"ID":"a98bb539-20d1-47bb-a093-b61c84d58289","Type":"ContainerDied","Data":"d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f"}
Jul 06 23:20:29.637788 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.637583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t" event={"ID":"a98bb539-20d1-47bb-a093-b61c84d58289","Type":"ContainerDied","Data":"efb18a21f0efdeb6b7ed14300435f8d538b2e46029946a34d87eb993f9ce5f7d"}
Jul 06 23:20:29.637788 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.637607    2563 scope.go:117] "RemoveContainer" containerID="d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f"
Jul 06 23:20:29.647544 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.647519    2563 scope.go:117] "RemoveContainer" containerID="d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f"
Jul 06 23:20:29.647904 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:20:29.647883    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f\": container with ID starting with d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f not found: ID does not exist" containerID="d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f"
Jul 06 23:20:29.647989 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.647913    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f"} err="failed to get container status \"d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f\": rpc error: code = NotFound desc = could not find container \"d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f\": container with ID starting with d3867240c19384f58d815d9613718fbec7741341e05ba298016a5ea0795ac01f not found: ID does not exist"
Jul 06 23:20:29.660440 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.660404    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"]
Jul 06 23:20:29.664547 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:29.664514    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f9d5b5-b9fd95fd6-6mg9t"]
Jul 06 23:20:30.962947 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:30.962911    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a98bb539-20d1-47bb-a093-b61c84d58289" path="/var/lib/kubelet/pods/a98bb539-20d1-47bb-a093-b61c84d58289/volumes"
Jul 06 23:20:38.724130 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.724091    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"]
Jul 06 23:20:38.724656 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.724565    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a98bb539-20d1-47bb-a093-b61c84d58289" containerName="maas-api"
Jul 06 23:20:38.724656 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.724586    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a98bb539-20d1-47bb-a093-b61c84d58289" containerName="maas-api"
Jul 06 23:20:38.724656 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.724614    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83230b6b-9338-4887-b60a-c916ff4410be" containerName="maas-api"
Jul 06 23:20:38.724656 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.724622    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="83230b6b-9338-4887-b60a-c916ff4410be" containerName="maas-api"
Jul 06 23:20:38.724882 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.724793    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="83230b6b-9338-4887-b60a-c916ff4410be" containerName="maas-api"
Jul 06 23:20:38.724882 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.724808    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a98bb539-20d1-47bb-a093-b61c84d58289" containerName="maas-api"
Jul 06 23:20:38.729522 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.729499    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:38.732053 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.732030    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-cb393d-serving-cert\""
Jul 06 23:20:38.739066 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.739041    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"]
Jul 06 23:20:38.884133 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.884100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b998f19d-4398-4974-9839-61ebba77ffd3-maas-api-tls\") pod \"maas-api-e2e-api-a-cb393d-5b447f5987-rpgww\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:38.884309 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.884168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998f19d-4398-4974-9839-61ebba77ffd3-openshift-service-ca\") pod \"maas-api-e2e-api-a-cb393d-5b447f5987-rpgww\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:38.884309 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.884189    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kgzg\" (UniqueName: \"kubernetes.io/projected/b998f19d-4398-4974-9839-61ebba77ffd3-kube-api-access-7kgzg\") pod \"maas-api-e2e-api-a-cb393d-5b447f5987-rpgww\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:38.984730 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.984619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b998f19d-4398-4974-9839-61ebba77ffd3-maas-api-tls\") pod \"maas-api-e2e-api-a-cb393d-5b447f5987-rpgww\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:38.984730 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.984722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998f19d-4398-4974-9839-61ebba77ffd3-openshift-service-ca\") pod \"maas-api-e2e-api-a-cb393d-5b447f5987-rpgww\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:38.984918 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.984743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kgzg\" (UniqueName: \"kubernetes.io/projected/b998f19d-4398-4974-9839-61ebba77ffd3-kube-api-access-7kgzg\") pod \"maas-api-e2e-api-a-cb393d-5b447f5987-rpgww\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:38.985539 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.985462    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998f19d-4398-4974-9839-61ebba77ffd3-openshift-service-ca\") pod \"maas-api-e2e-api-a-cb393d-5b447f5987-rpgww\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:38.987284 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.987260    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b998f19d-4398-4974-9839-61ebba77ffd3-maas-api-tls\") pod \"maas-api-e2e-api-a-cb393d-5b447f5987-rpgww\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:38.994022 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:38.993990    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kgzg\" (UniqueName: \"kubernetes.io/projected/b998f19d-4398-4974-9839-61ebba77ffd3-kube-api-access-7kgzg\") pod \"maas-api-e2e-api-a-cb393d-5b447f5987-rpgww\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:39.041039 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:39.040997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:39.194318 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:39.194293    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"]
Jul 06 23:20:39.196823 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:20:39.196796    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb998f19d_4398_4974_9839_61ebba77ffd3.slice/crio-5851ff907ae47759de79cd690a9e7c5918bc31ec4d0d6a76912053d12b5c65cb WatchSource:0}: Error finding container 5851ff907ae47759de79cd690a9e7c5918bc31ec4d0d6a76912053d12b5c65cb: Status 404 returned error can't find the container with id 5851ff907ae47759de79cd690a9e7c5918bc31ec4d0d6a76912053d12b5c65cb
Jul 06 23:20:39.685465 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:39.685428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww" event={"ID":"b998f19d-4398-4974-9839-61ebba77ffd3","Type":"ContainerStarted","Data":"449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49"}
Jul 06 23:20:39.685632 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:39.685483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww" event={"ID":"b998f19d-4398-4974-9839-61ebba77ffd3","Type":"ContainerStarted","Data":"5851ff907ae47759de79cd690a9e7c5918bc31ec4d0d6a76912053d12b5c65cb"}
Jul 06 23:20:39.685632 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:39.685506    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:39.706318 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:39.706259    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww" podStartSLOduration=1.376532562 podStartE2EDuration="1.706240488s" totalImagesPullingTime="329.707926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:20:39.705094042 +0000 UTC m=+2061.247425215" watchObservedRunningTime="2026-07-06 23:20:39.706240488 +0000 UTC m=+2061.248571658"
Jul 06 23:20:45.695023 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:45.694993    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:20:46.772455 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:46.772410    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"]
Jul 06 23:20:46.776142 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:46.776117    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:46.778938 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:46.778911    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ef756c-serving-cert\""
Jul 06 23:20:46.793540 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:46.793506    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"]
Jul 06 23:20:46.956608 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:46.956574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnr8q\" (UniqueName: \"kubernetes.io/projected/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-kube-api-access-vnr8q\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:46.956821 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:46.956627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:46.956821 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:46.956660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-openshift-service-ca\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:47.057784 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:47.057665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnr8q\" (UniqueName: \"kubernetes.io/projected/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-kube-api-access-vnr8q\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:47.057784 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:47.057763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:47.057784 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:47.057783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-openshift-service-ca\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:47.058068 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:20:47.057921    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-serving-cert: secret "maas-api-e2e-api-b-ef756c-serving-cert" not found
Jul 06 23:20:47.058068 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:20:47.057995    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls podName:5fc9036d-cac7-4864-9e9e-f428dd4e8b34 nodeName:}" failed. No retries permitted until 2026-07-06 23:20:47.55797302 +0000 UTC m=+2069.100304187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls") pod "maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc" (UID: "5fc9036d-cac7-4864-9e9e-f428dd4e8b34") : secret "maas-api-e2e-api-b-ef756c-serving-cert" not found
Jul 06 23:20:47.058388 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:47.058370    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-openshift-service-ca\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:47.068081 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:47.068053    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnr8q\" (UniqueName: \"kubernetes.io/projected/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-kube-api-access-vnr8q\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:47.563497 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:47.563463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:47.563738 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:20:47.563658    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-serving-cert: secret "maas-api-e2e-api-b-ef756c-serving-cert" not found
Jul 06 23:20:47.563821 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:20:47.563744    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls podName:5fc9036d-cac7-4864-9e9e-f428dd4e8b34 nodeName:}" failed. No retries permitted until 2026-07-06 23:20:48.563718724 +0000 UTC m=+2070.106049877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls") pod "maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc" (UID: "5fc9036d-cac7-4864-9e9e-f428dd4e8b34") : secret "maas-api-e2e-api-b-ef756c-serving-cert" not found
Jul 06 23:20:48.573860 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:48.573824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:48.576397 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:48.576370    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls\") pod \"maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:48.587774 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:48.587746    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:48.726893 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:48.726868    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"]
Jul 06 23:20:48.729308 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:20:48.729279    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc9036d_cac7_4864_9e9e_f428dd4e8b34.slice/crio-a538a187fda9687b608f9aed6133663b79cedc3a22172f1db2369cde07c03f3d WatchSource:0}: Error finding container a538a187fda9687b608f9aed6133663b79cedc3a22172f1db2369cde07c03f3d: Status 404 returned error can't find the container with id a538a187fda9687b608f9aed6133663b79cedc3a22172f1db2369cde07c03f3d
Jul 06 23:20:49.729539 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:49.729505    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc" event={"ID":"5fc9036d-cac7-4864-9e9e-f428dd4e8b34","Type":"ContainerStarted","Data":"fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568"}
Jul 06 23:20:49.729990 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:49.729554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc" event={"ID":"5fc9036d-cac7-4864-9e9e-f428dd4e8b34","Type":"ContainerStarted","Data":"a538a187fda9687b608f9aed6133663b79cedc3a22172f1db2369cde07c03f3d"}
Jul 06 23:20:49.729990 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:49.729607    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:49.748331 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:49.748277    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc" podStartSLOduration=3.414967231 podStartE2EDuration="3.748257666s" totalImagesPullingTime="333.290435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:20:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:20:49.747036588 +0000 UTC m=+2071.289367759" watchObservedRunningTime="2026-07-06 23:20:49.748257666 +0000 UTC m=+2071.290588836"
Jul 06 23:20:55.739526 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:55.739493    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:20:59.737306 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:59.737275    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"]
Jul 06 23:20:59.737826 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:59.737486    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww" podUID="b998f19d-4398-4974-9839-61ebba77ffd3" containerName="maas-api" containerID="cri-o://449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49" gracePeriod=30
Jul 06 23:20:59.989747 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:20:59.989678    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:21:00.079238 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.079195    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kgzg\" (UniqueName: \"kubernetes.io/projected/b998f19d-4398-4974-9839-61ebba77ffd3-kube-api-access-7kgzg\") pod \"b998f19d-4398-4974-9839-61ebba77ffd3\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") "
Jul 06 23:21:00.079435 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.079300    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998f19d-4398-4974-9839-61ebba77ffd3-openshift-service-ca\") pod \"b998f19d-4398-4974-9839-61ebba77ffd3\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") "
Jul 06 23:21:00.079435 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.079379    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b998f19d-4398-4974-9839-61ebba77ffd3-maas-api-tls\") pod \"b998f19d-4398-4974-9839-61ebba77ffd3\" (UID: \"b998f19d-4398-4974-9839-61ebba77ffd3\") "
Jul 06 23:21:00.079746 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.079715    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b998f19d-4398-4974-9839-61ebba77ffd3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b998f19d-4398-4974-9839-61ebba77ffd3" (UID: "b998f19d-4398-4974-9839-61ebba77ffd3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:21:00.081574 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.081550    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b998f19d-4398-4974-9839-61ebba77ffd3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b998f19d-4398-4974-9839-61ebba77ffd3" (UID: "b998f19d-4398-4974-9839-61ebba77ffd3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:21:00.081844 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.081820    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b998f19d-4398-4974-9839-61ebba77ffd3-kube-api-access-7kgzg" (OuterVolumeSpecName: "kube-api-access-7kgzg") pod "b998f19d-4398-4974-9839-61ebba77ffd3" (UID: "b998f19d-4398-4974-9839-61ebba77ffd3"). InnerVolumeSpecName "kube-api-access-7kgzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:21:00.180559 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.180516    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b998f19d-4398-4974-9839-61ebba77ffd3-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:21:00.180559 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.180549    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kgzg\" (UniqueName: \"kubernetes.io/projected/b998f19d-4398-4974-9839-61ebba77ffd3-kube-api-access-7kgzg\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:21:00.180559 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.180561    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998f19d-4398-4974-9839-61ebba77ffd3-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:21:00.780679 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.780615    2563 generic.go:356] "Generic (PLEG): container finished" podID="b998f19d-4398-4974-9839-61ebba77ffd3" containerID="449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49" exitCode=0
Jul 06 23:21:00.780679 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.780679    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"
Jul 06 23:21:00.781218 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.780700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww" event={"ID":"b998f19d-4398-4974-9839-61ebba77ffd3","Type":"ContainerDied","Data":"449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49"}
Jul 06 23:21:00.781218 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.780753    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww" event={"ID":"b998f19d-4398-4974-9839-61ebba77ffd3","Type":"ContainerDied","Data":"5851ff907ae47759de79cd690a9e7c5918bc31ec4d0d6a76912053d12b5c65cb"}
Jul 06 23:21:00.781218 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.780775    2563 scope.go:117] "RemoveContainer" containerID="449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49"
Jul 06 23:21:00.789470 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.789454    2563 scope.go:117] "RemoveContainer" containerID="449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49"
Jul 06 23:21:00.789800 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:21:00.789773    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49\": container with ID starting with 449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49 not found: ID does not exist" containerID="449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49"
Jul 06 23:21:00.789893 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.789811    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49"} err="failed to get container status \"449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49\": rpc error: code = NotFound desc = could not find container \"449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49\": container with ID starting with 449652d13dace648b414c2d24624a43b2426ab21c7960b5ba7d20d097619fe49 not found: ID does not exist"
Jul 06 23:21:00.801539 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.801514    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"]
Jul 06 23:21:00.805408 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.805386    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb393d-5b447f5987-rpgww"]
Jul 06 23:21:00.963837 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:00.963807    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b998f19d-4398-4974-9839-61ebba77ffd3" path="/var/lib/kubelet/pods/b998f19d-4398-4974-9839-61ebba77ffd3/volumes"
Jul 06 23:21:06.781711 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:06.781676    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"]
Jul 06 23:21:06.782115 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:06.781881    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc" podUID="5fc9036d-cac7-4864-9e9e-f428dd4e8b34" containerName="maas-api" containerID="cri-o://fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568" gracePeriod=30
Jul 06 23:21:07.024872 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.024847    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:21:07.139343 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.139251    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-openshift-service-ca\") pod \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") "
Jul 06 23:21:07.139490 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.139354    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls\") pod \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") "
Jul 06 23:21:07.139490 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.139394    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnr8q\" (UniqueName: \"kubernetes.io/projected/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-kube-api-access-vnr8q\") pod \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\" (UID: \"5fc9036d-cac7-4864-9e9e-f428dd4e8b34\") "
Jul 06 23:21:07.139722 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.139702    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5fc9036d-cac7-4864-9e9e-f428dd4e8b34" (UID: "5fc9036d-cac7-4864-9e9e-f428dd4e8b34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:21:07.141610 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.141579    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-kube-api-access-vnr8q" (OuterVolumeSpecName: "kube-api-access-vnr8q") pod "5fc9036d-cac7-4864-9e9e-f428dd4e8b34" (UID: "5fc9036d-cac7-4864-9e9e-f428dd4e8b34"). InnerVolumeSpecName "kube-api-access-vnr8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:21:07.141610 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.141604    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5fc9036d-cac7-4864-9e9e-f428dd4e8b34" (UID: "5fc9036d-cac7-4864-9e9e-f428dd4e8b34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:21:07.240017 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.239982    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:21:07.240017 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.240009    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnr8q\" (UniqueName: \"kubernetes.io/projected/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-kube-api-access-vnr8q\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:21:07.240017 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.240018    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fc9036d-cac7-4864-9e9e-f428dd4e8b34-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:21:07.813471 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.813432    2563 generic.go:356] "Generic (PLEG): container finished" podID="5fc9036d-cac7-4864-9e9e-f428dd4e8b34" containerID="fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568" exitCode=0
Jul 06 23:21:07.813916 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.813481    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"
Jul 06 23:21:07.813916 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.813514    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc" event={"ID":"5fc9036d-cac7-4864-9e9e-f428dd4e8b34","Type":"ContainerDied","Data":"fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568"}
Jul 06 23:21:07.813916 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.813568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc" event={"ID":"5fc9036d-cac7-4864-9e9e-f428dd4e8b34","Type":"ContainerDied","Data":"a538a187fda9687b608f9aed6133663b79cedc3a22172f1db2369cde07c03f3d"}
Jul 06 23:21:07.813916 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.813590    2563 scope.go:117] "RemoveContainer" containerID="fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568"
Jul 06 23:21:07.823159 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.823141    2563 scope.go:117] "RemoveContainer" containerID="fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568"
Jul 06 23:21:07.823434 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:21:07.823416    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568\": container with ID starting with fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568 not found: ID does not exist" containerID="fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568"
Jul 06 23:21:07.823472 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.823445    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568"} err="failed to get container status \"fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568\": rpc error: code = NotFound desc = could not find container \"fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568\": container with ID starting with fdedaf4754c83b67ad5c4639aaabccc5693721e1e756f9e57a882ad707772568 not found: ID does not exist"
Jul 06 23:21:07.834602 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.834576    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"]
Jul 06 23:21:07.839843 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:07.839820    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ef756c-85c69bf976-kwdnc"]
Jul 06 23:21:08.962426 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:08.962388    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fc9036d-cac7-4864-9e9e-f428dd4e8b34" path="/var/lib/kubelet/pods/5fc9036d-cac7-4864-9e9e-f428dd4e8b34/volumes"
Jul 06 23:21:17.079959 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.079922    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"]
Jul 06 23:21:17.080356 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.080228    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fc9036d-cac7-4864-9e9e-f428dd4e8b34" containerName="maas-api"
Jul 06 23:21:17.080356 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.080239    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fc9036d-cac7-4864-9e9e-f428dd4e8b34" containerName="maas-api"
Jul 06 23:21:17.080356 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.080252    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b998f19d-4398-4974-9839-61ebba77ffd3" containerName="maas-api"
Jul 06 23:21:17.080356 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.080258    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b998f19d-4398-4974-9839-61ebba77ffd3" containerName="maas-api"
Jul 06 23:21:17.080356 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.080309    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fc9036d-cac7-4864-9e9e-f428dd4e8b34" containerName="maas-api"
Jul 06 23:21:17.080356 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.080318    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b998f19d-4398-4974-9839-61ebba77ffd3" containerName="maas-api"
Jul 06 23:21:17.084475 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.084456    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.087104 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.087080    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-2cc96a-serving-cert\""
Jul 06 23:21:17.093979 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.093954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"]
Jul 06 23:21:17.226035 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.225990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29c08c20-bc38-4572-aed5-d119f03b49fc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.226219 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.226045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz4c6\" (UniqueName: \"kubernetes.io/projected/29c08c20-bc38-4572-aed5-d119f03b49fc-kube-api-access-bz4c6\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.226219 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.226133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29c08c20-bc38-4572-aed5-d119f03b49fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.327221 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.327182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz4c6\" (UniqueName: \"kubernetes.io/projected/29c08c20-bc38-4572-aed5-d119f03b49fc-kube-api-access-bz4c6\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.327389 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.327231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29c08c20-bc38-4572-aed5-d119f03b49fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.327389 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.327285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29c08c20-bc38-4572-aed5-d119f03b49fc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.327538 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:21:17.327401    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-serving-cert: secret "maas-api-e2e-shared-a-2cc96a-serving-cert" not found
Jul 06 23:21:17.327538 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:21:17.327491    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c08c20-bc38-4572-aed5-d119f03b49fc-maas-api-tls podName:29c08c20-bc38-4572-aed5-d119f03b49fc nodeName:}" failed. No retries permitted until 2026-07-06 23:21:17.827468698 +0000 UTC m=+2099.369799850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/29c08c20-bc38-4572-aed5-d119f03b49fc-maas-api-tls") pod "maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn" (UID: "29c08c20-bc38-4572-aed5-d119f03b49fc") : secret "maas-api-e2e-shared-a-2cc96a-serving-cert" not found
Jul 06 23:21:17.327943 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.327924    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29c08c20-bc38-4572-aed5-d119f03b49fc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.341682 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.341585    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz4c6\" (UniqueName: \"kubernetes.io/projected/29c08c20-bc38-4572-aed5-d119f03b49fc-kube-api-access-bz4c6\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.831669 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.831609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29c08c20-bc38-4572-aed5-d119f03b49fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.834130 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.834106    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29c08c20-bc38-4572-aed5-d119f03b49fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:17.995691 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:17.995617    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:18.139282 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:18.139258    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"]
Jul 06 23:21:18.142128 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:21:18.142094    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c08c20_bc38_4572_aed5_d119f03b49fc.slice/crio-ec2e69dd1e961923df3b7f0ac03d3ca07bbd492c1813ba5904ffd8f31affe367 WatchSource:0}: Error finding container ec2e69dd1e961923df3b7f0ac03d3ca07bbd492c1813ba5904ffd8f31affe367: Status 404 returned error can't find the container with id ec2e69dd1e961923df3b7f0ac03d3ca07bbd492c1813ba5904ffd8f31affe367
Jul 06 23:21:18.863609 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:18.863515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn" event={"ID":"29c08c20-bc38-4572-aed5-d119f03b49fc","Type":"ContainerStarted","Data":"928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99"}
Jul 06 23:21:18.863609 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:18.863566    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn" event={"ID":"29c08c20-bc38-4572-aed5-d119f03b49fc","Type":"ContainerStarted","Data":"ec2e69dd1e961923df3b7f0ac03d3ca07bbd492c1813ba5904ffd8f31affe367"}
Jul 06 23:21:18.863609 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:18.863603    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:24.873631 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:24.873599    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:21:24.891693 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:24.891608    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn" podStartSLOduration=7.483443135 podStartE2EDuration="7.891590978s" totalImagesPullingTime="408.147843ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:21:18.902404093 +0000 UTC m=+2100.444735263" watchObservedRunningTime="2026-07-06 23:21:24.891590978 +0000 UTC m=+2106.433922147"
Jul 06 23:21:25.089474 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.089434    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"]
Jul 06 23:21:25.093267 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.093241    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.096094 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.096071    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ca352b-serving-cert\""
Jul 06 23:21:25.107810 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.107751    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"]
Jul 06 23:21:25.198173 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.198135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zjsf\" (UniqueName: \"kubernetes.io/projected/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-kube-api-access-8zjsf\") pod \"maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.198368 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.198205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-maas-api-tls\") pod \"maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.198368 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.198288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.298922 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.298883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-maas-api-tls\") pod \"maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.299109 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.298938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.299109 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.298970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zjsf\" (UniqueName: \"kubernetes.io/projected/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-kube-api-access-8zjsf\") pod \"maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.299756 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.299729    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.301823 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.301793    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-maas-api-tls\") pod \"maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.312862 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.312839    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zjsf\" (UniqueName: \"kubernetes.io/projected/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-kube-api-access-8zjsf\") pod \"maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.405690 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.405658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:25.556005 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.555873    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"]
Jul 06 23:21:25.558748 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:21:25.558718    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2e1cb4_b31f_4c4d_ad98_1763b5dc12af.slice/crio-a1dd12c17c9313b3b0f6c12686006cdcb7c931e09e34ef97289f813165aaee34 WatchSource:0}: Error finding container a1dd12c17c9313b3b0f6c12686006cdcb7c931e09e34ef97289f813165aaee34: Status 404 returned error can't find the container with id a1dd12c17c9313b3b0f6c12686006cdcb7c931e09e34ef97289f813165aaee34
Jul 06 23:21:25.894110 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:25.894021    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz" event={"ID":"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af","Type":"ContainerStarted","Data":"a1dd12c17c9313b3b0f6c12686006cdcb7c931e09e34ef97289f813165aaee34"}
Jul 06 23:21:26.899742 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:26.899703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz" event={"ID":"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af","Type":"ContainerStarted","Data":"309f9e30192bc8d96b5fa624044f2e603c7f74cc6908d4accb5556e1879339f1"}
Jul 06 23:21:26.900110 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:26.899876    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:21:26.925682 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:26.925605    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz" podStartSLOduration=1.561596518 podStartE2EDuration="1.925587806s" totalImagesPullingTime="363.991288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:21:26.924957267 +0000 UTC m=+2108.467288438" watchObservedRunningTime="2026-07-06 23:21:26.925587806 +0000 UTC m=+2108.467918976"
Jul 06 23:21:32.909034 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:21:32.909003    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:27:23.631230 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:23.631195    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"]
Jul 06 23:27:23.631856 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:23.631510    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn" podUID="29c08c20-bc38-4572-aed5-d119f03b49fc" containerName="maas-api" containerID="cri-o://928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99" gracePeriod=30
Jul 06 23:27:24.076759 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.076734    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:27:24.095847 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.095816    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz4c6\" (UniqueName: \"kubernetes.io/projected/29c08c20-bc38-4572-aed5-d119f03b49fc-kube-api-access-bz4c6\") pod \"29c08c20-bc38-4572-aed5-d119f03b49fc\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") "
Jul 06 23:27:24.096018 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.095868    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29c08c20-bc38-4572-aed5-d119f03b49fc-openshift-service-ca\") pod \"29c08c20-bc38-4572-aed5-d119f03b49fc\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") "
Jul 06 23:27:24.096018 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.095905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29c08c20-bc38-4572-aed5-d119f03b49fc-maas-api-tls\") pod \"29c08c20-bc38-4572-aed5-d119f03b49fc\" (UID: \"29c08c20-bc38-4572-aed5-d119f03b49fc\") "
Jul 06 23:27:24.096273 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.096249    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c08c20-bc38-4572-aed5-d119f03b49fc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29c08c20-bc38-4572-aed5-d119f03b49fc" (UID: "29c08c20-bc38-4572-aed5-d119f03b49fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:27:24.098250 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.098223    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c08c20-bc38-4572-aed5-d119f03b49fc-kube-api-access-bz4c6" (OuterVolumeSpecName: "kube-api-access-bz4c6") pod "29c08c20-bc38-4572-aed5-d119f03b49fc" (UID: "29c08c20-bc38-4572-aed5-d119f03b49fc"). InnerVolumeSpecName "kube-api-access-bz4c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:27:24.098355 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.098259    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29c08c20-bc38-4572-aed5-d119f03b49fc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29c08c20-bc38-4572-aed5-d119f03b49fc" (UID: "29c08c20-bc38-4572-aed5-d119f03b49fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:27:24.196811 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.196781    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz4c6\" (UniqueName: \"kubernetes.io/projected/29c08c20-bc38-4572-aed5-d119f03b49fc-kube-api-access-bz4c6\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:27:24.196811 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.196807    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29c08c20-bc38-4572-aed5-d119f03b49fc-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:27:24.196811 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.196818    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29c08c20-bc38-4572-aed5-d119f03b49fc-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:27:24.450618 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.450523    2563 generic.go:356] "Generic (PLEG): container finished" podID="29c08c20-bc38-4572-aed5-d119f03b49fc" containerID="928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99" exitCode=0
Jul 06 23:27:24.450618 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.450574    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"
Jul 06 23:27:24.450618 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.450591    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn" event={"ID":"29c08c20-bc38-4572-aed5-d119f03b49fc","Type":"ContainerDied","Data":"928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99"}
Jul 06 23:27:24.450846 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.450673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn" event={"ID":"29c08c20-bc38-4572-aed5-d119f03b49fc","Type":"ContainerDied","Data":"ec2e69dd1e961923df3b7f0ac03d3ca07bbd492c1813ba5904ffd8f31affe367"}
Jul 06 23:27:24.450846 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.450706    2563 scope.go:117] "RemoveContainer" containerID="928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99"
Jul 06 23:27:24.459719 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.459630    2563 scope.go:117] "RemoveContainer" containerID="928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99"
Jul 06 23:27:24.460000 ip-10-0-140-53 kubenswrapper[2563]: E0706 23:27:24.459984    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99\": container with ID starting with 928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99 not found: ID does not exist" containerID="928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99"
Jul 06 23:27:24.460052 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.460009    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99"} err="failed to get container status \"928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99\": rpc error: code = NotFound desc = could not find container \"928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99\": container with ID starting with 928f6eefb09c9355006b798bb244ce6c8d026b98cdcb8e854ab01463c5ad5e99 not found: ID does not exist"
Jul 06 23:27:24.471404 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.471373    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"]
Jul 06 23:27:24.473175 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.473153    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2cc96a-58b544f79d-9bmsn"]
Jul 06 23:27:24.963837 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:24.963794    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29c08c20-bc38-4572-aed5-d119f03b49fc" path="/var/lib/kubelet/pods/29c08c20-bc38-4572-aed5-d119f03b49fc/volumes"
Jul 06 23:27:43.335444 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.335409    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"]
Jul 06 23:27:43.335923 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.335609    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz" podUID="8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af" containerName="maas-api" containerID="cri-o://309f9e30192bc8d96b5fa624044f2e603c7f74cc6908d4accb5556e1879339f1" gracePeriod=30
Jul 06 23:27:43.535023 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.534986    2563 generic.go:356] "Generic (PLEG): container finished" podID="8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af" containerID="309f9e30192bc8d96b5fa624044f2e603c7f74cc6908d4accb5556e1879339f1" exitCode=0
Jul 06 23:27:43.535196 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.535058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz" event={"ID":"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af","Type":"ContainerDied","Data":"309f9e30192bc8d96b5fa624044f2e603c7f74cc6908d4accb5556e1879339f1"}
Jul 06 23:27:43.577552 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.577527    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:27:43.670029 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.669943    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-maas-api-tls\") pod \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") "
Jul 06 23:27:43.670029 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.669984    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-openshift-service-ca\") pod \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") "
Jul 06 23:27:43.670029 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.670029    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zjsf\" (UniqueName: \"kubernetes.io/projected/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-kube-api-access-8zjsf\") pod \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\" (UID: \"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af\") "
Jul 06 23:27:43.670382 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.670359    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af" (UID: "8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:27:43.672252 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.672224    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af" (UID: "8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:27:43.672354 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.672268    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-kube-api-access-8zjsf" (OuterVolumeSpecName: "kube-api-access-8zjsf") pod "8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af" (UID: "8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af"). InnerVolumeSpecName "kube-api-access-8zjsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:27:43.770815 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.770777    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-maas-api-tls\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:27:43.770815 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.770807    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-openshift-service-ca\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:27:43.770815 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:43.770818    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zjsf\" (UniqueName: \"kubernetes.io/projected/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af-kube-api-access-8zjsf\") on node \"ip-10-0-140-53.ec2.internal\" DevicePath \"\""
Jul 06 23:27:44.541727 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:44.541694    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"
Jul 06 23:27:44.541727 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:44.541712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz" event={"ID":"8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af","Type":"ContainerDied","Data":"a1dd12c17c9313b3b0f6c12686006cdcb7c931e09e34ef97289f813165aaee34"}
Jul 06 23:27:44.542244 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:44.541782    2563 scope.go:117] "RemoveContainer" containerID="309f9e30192bc8d96b5fa624044f2e603c7f74cc6908d4accb5556e1879339f1"
Jul 06 23:27:44.562442 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:44.562406    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"]
Jul 06 23:27:44.566243 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:44.566217    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ca352b-5fb7868dd5-k9cbz"]
Jul 06 23:27:44.963150 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:44.963116    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af" path="/var/lib/kubelet/pods/8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af/volumes"
Jul 06 23:27:54.369294 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:54.369257    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f58865b75-l95kx_a08adfff-5ed0-4ea7-8bd6-d907bab56828/manager/0.log"
Jul 06 23:27:54.822213 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:54.822184    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84df5575d5-jjjbs_5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc/copy-manifests/0.log"
Jul 06 23:27:54.834157 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:54.834122    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84df5575d5-jjjbs_5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc/manager/0.log"
Jul 06 23:27:57.432865 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:57.432832    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pj6sf_6be26566-201d-448c-9f40-70a67e14c894/discovery/0.log"
Jul 06 23:27:57.885970 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:57.885883    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5rm74_b90e7886-4b88-4923-a310-61d4c20d28e8/payload-pre-processing/0.log"
Jul 06 23:27:58.458543 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:58.458497    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-tht5h_c380c5d0-99cb-4295-a43a-22ba73d4557b/main/0.log"
Jul 06 23:27:58.788579 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:27:58.788504    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxs92b_da761aac-d953-4196-8661-15fa1e055977/main/0.log"
Jul 06 23:28:05.616523 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:05.616487    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mgsdw_d7e29fc5-f57a-45bd-b114-f48b56ea7b6c/global-pull-secret-syncer/0.log"
Jul 06 23:28:05.776785 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:05.776755    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7nkqp_b5403f15-1e81-4b5d-b409-6b18e29e201f/konnectivity-agent/0.log"
Jul 06 23:28:05.873113 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:05.873031    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-53.ec2.internal_3383faa9d76785b568517c84a7fb072b/haproxy/0.log"
Jul 06 23:28:13.106240 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:13.106157    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-4czjz_feaa18c2-daed-4b96-8c5d-ddb3b8a4cb95/cluster-monitoring-operator/0.log"
Jul 06 23:28:13.483075 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:13.483045    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z7tdr_a28f9b11-6d92-42a1-a55b-28d8211f6ed2/node-exporter/0.log"
Jul 06 23:28:13.506615 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:13.506587    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z7tdr_a28f9b11-6d92-42a1-a55b-28d8211f6ed2/kube-rbac-proxy/0.log"
Jul 06 23:28:13.531213 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:13.531183    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z7tdr_a28f9b11-6d92-42a1-a55b-28d8211f6ed2/init-textfile/0.log"
Jul 06 23:28:16.222782 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.222751    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"]
Jul 06 23:28:16.223213 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.223069    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af" containerName="maas-api"
Jul 06 23:28:16.223213 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.223080    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af" containerName="maas-api"
Jul 06 23:28:16.223213 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.223098    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29c08c20-bc38-4572-aed5-d119f03b49fc" containerName="maas-api"
Jul 06 23:28:16.223213 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.223103    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="29c08c20-bc38-4572-aed5-d119f03b49fc" containerName="maas-api"
Jul 06 23:28:16.223213 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.223148    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="29c08c20-bc38-4572-aed5-d119f03b49fc" containerName="maas-api"
Jul 06 23:28:16.223213 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.223157    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e2e1cb4-b31f-4c4d-ad98-1763b5dc12af" containerName="maas-api"
Jul 06 23:28:16.226082 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.226067    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.228499 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.228465    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h8t4j\"/\"openshift-service-ca.crt\""
Jul 06 23:28:16.228666 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.228586    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h8t4j\"/\"kube-root-ca.crt\""
Jul 06 23:28:16.229415 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.229394    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h8t4j\"/\"default-dockercfg-csq5b\""
Jul 06 23:28:16.235815 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.235784    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"]
Jul 06 23:28:16.251325 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.251296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-proc\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.251468 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.251335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-sys\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.251468 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.251361    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-podres\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.251468 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.251427    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-lib-modules\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.251571 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.251469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptp26\" (UniqueName: \"kubernetes.io/projected/b2428c02-5e69-47d4-8165-3ce3740c5ff6-kube-api-access-ptp26\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.352695 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.352621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptp26\" (UniqueName: \"kubernetes.io/projected/b2428c02-5e69-47d4-8165-3ce3740c5ff6-kube-api-access-ptp26\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.352874 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.352737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-proc\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.352874 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.352768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-sys\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.352874 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.352786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-podres\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.352874 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.352811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-lib-modules\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.353030 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.352879    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-sys\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.353030 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.352879    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-proc\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.353030 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.352950    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-podres\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.353030 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.352950    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2428c02-5e69-47d4-8165-3ce3740c5ff6-lib-modules\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.360600 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.360570    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptp26\" (UniqueName: \"kubernetes.io/projected/b2428c02-5e69-47d4-8165-3ce3740c5ff6-kube-api-access-ptp26\") pod \"perf-node-gather-daemonset-b8sn9\" (UID: \"b2428c02-5e69-47d4-8165-3ce3740c5ff6\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.537127 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.537028    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:16.669586 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.669563    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"]
Jul 06 23:28:16.671682 ip-10-0-140-53 kubenswrapper[2563]: W0706 23:28:16.671657    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb2428c02_5e69_47d4_8165_3ce3740c5ff6.slice/crio-68bbe17975ac33cef992fc9dd2b203fa719b0aed3eb84361d558573b9972914c WatchSource:0}: Error finding container 68bbe17975ac33cef992fc9dd2b203fa719b0aed3eb84361d558573b9972914c: Status 404 returned error can't find the container with id 68bbe17975ac33cef992fc9dd2b203fa719b0aed3eb84361d558573b9972914c
Jul 06 23:28:16.682059 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:16.682029    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9" event={"ID":"b2428c02-5e69-47d4-8165-3ce3740c5ff6","Type":"ContainerStarted","Data":"68bbe17975ac33cef992fc9dd2b203fa719b0aed3eb84361d558573b9972914c"}
Jul 06 23:28:17.688189 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:17.688129    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9" event={"ID":"b2428c02-5e69-47d4-8165-3ce3740c5ff6","Type":"ContainerStarted","Data":"f71316956bc4711b9a541f307214fbfe091ab10e8cf53021655813fa4f1116de"}
Jul 06 23:28:17.688582 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:17.688245    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:17.709096 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:17.709040    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9" podStartSLOduration=1.7090243840000001 podStartE2EDuration="1.709024384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:28:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:28:17.707418051 +0000 UTC m=+2519.249749247" watchObservedRunningTime="2026-07-06 23:28:17.709024384 +0000 UTC m=+2519.251355553"
Jul 06 23:28:18.292805 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:18.292776    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4dnvj_b615cfe7-ddc3-4a28-8c5a-5e0fe646960a/dns/0.log"
Jul 06 23:28:18.315651 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:18.315608    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4dnvj_b615cfe7-ddc3-4a28-8c5a-5e0fe646960a/kube-rbac-proxy/0.log"
Jul 06 23:28:18.483524 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:18.483492    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n6jp9_fa35364e-53e3-4b89-a7bf-584d76ce3005/dns-node-resolver/0.log"
Jul 06 23:28:19.067583 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:19.067557    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qhn45_b05d5380-0b82-4ade-9481-2dcd94f82354/node-ca/0.log"
Jul 06 23:28:20.149766 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:20.149733    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pj6sf_6be26566-201d-448c-9f40-70a67e14c894/discovery/0.log"
Jul 06 23:28:20.290594 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:20.290564    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5rm74_b90e7886-4b88-4923-a310-61d4c20d28e8/payload-pre-processing/0.log"
Jul 06 23:28:21.037574 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:21.037539    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2jb5z_2d80dea7-f05c-4ea3-ad20-66f0645412af/serve-healthcheck-canary/0.log"
Jul 06 23:28:21.564360 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:21.564331    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5bnck_4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99/kube-rbac-proxy/0.log"
Jul 06 23:28:21.584562 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:21.584537    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5bnck_4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99/exporter/0.log"
Jul 06 23:28:21.604536 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:21.604510    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5bnck_4c2c3fa4-94c6-4be8-a2fc-ffd3f796cc99/extractor/0.log"
Jul 06 23:28:23.702736 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:23.702702    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-b8sn9"
Jul 06 23:28:24.180348 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:24.180314    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f58865b75-l95kx_a08adfff-5ed0-4ea7-8bd6-d907bab56828/manager/0.log"
Jul 06 23:28:24.447804 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:24.447723    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84df5575d5-jjjbs_5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc/manager/0.log"
Jul 06 23:28:24.466216 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:24.466191    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84df5575d5-jjjbs_5fddc9b8-8b37-4bb9-bd49-50aa0a5578fc/copy-manifests/0.log"
Jul 06 23:28:25.910244 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:25.910221    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5747d5c666-g6gkg_30588141-2216-45d3-9dac-958c26affe46/manager/0.log"
Jul 06 23:28:27.124753 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:27.124721    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pj6sf_6be26566-201d-448c-9f40-70a67e14c894/discovery/0.log"
Jul 06 23:28:27.233105 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:27.233080    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5rm74_b90e7886-4b88-4923-a310-61d4c20d28e8/payload-pre-processing/0.log"
Jul 06 23:28:28.621523 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:28.621472    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pj6sf_6be26566-201d-448c-9f40-70a67e14c894/discovery/0.log"
Jul 06 23:28:28.732078 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:28.732052    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-5rm74_b90e7886-4b88-4923-a310-61d4c20d28e8/payload-pre-processing/0.log"
Jul 06 23:28:31.341194 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:31.341164    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-dn2sz_87a34b5b-effd-46cd-b1d9-26d0e53630f5/migrator/0.log"
Jul 06 23:28:31.363070 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:31.363044    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-dn2sz_87a34b5b-effd-46cd-b1d9-26d0e53630f5/graceful-termination/0.log"
Jul 06 23:28:31.735187 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:31.735153    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-6r9bb_fc071d16-fade-4e10-9e05-b5269d503961/kube-storage-version-migrator-operator/1.log"
Jul 06 23:28:31.736048 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:31.736031    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-6r9bb_fc071d16-fade-4e10-9e05-b5269d503961/kube-storage-version-migrator-operator/0.log"
Jul 06 23:28:32.838218 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:32.838190    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mk4bk_f2a8421b-dbce-413f-b6ea-d4c15efb2dc1/kube-multus-additional-cni-plugins/0.log"
Jul 06 23:28:32.857367 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:32.857337    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mk4bk_f2a8421b-dbce-413f-b6ea-d4c15efb2dc1/egress-router-binary-copy/0.log"
Jul 06 23:28:32.876188 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:32.876158    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mk4bk_f2a8421b-dbce-413f-b6ea-d4c15efb2dc1/cni-plugins/0.log"
Jul 06 23:28:32.894840 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:32.894812    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mk4bk_f2a8421b-dbce-413f-b6ea-d4c15efb2dc1/bond-cni-plugin/0.log"
Jul 06 23:28:32.913323 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:32.913297    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mk4bk_f2a8421b-dbce-413f-b6ea-d4c15efb2dc1/routeoverride-cni/0.log"
Jul 06 23:28:32.933504 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:32.933476    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mk4bk_f2a8421b-dbce-413f-b6ea-d4c15efb2dc1/whereabouts-cni-bincopy/0.log"
Jul 06 23:28:32.952269 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:32.952240    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mk4bk_f2a8421b-dbce-413f-b6ea-d4c15efb2dc1/whereabouts-cni/0.log"
Jul 06 23:28:33.167326 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:33.167250    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pmslq_bba4f2f3-ca52-4cb3-a21d-830bfc6815c2/kube-multus/0.log"
Jul 06 23:28:33.186723 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:33.186694    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4jvqw_b8a3215b-b208-420a-82f1-4907c79efac1/network-metrics-daemon/0.log"
Jul 06 23:28:33.206145 ip-10-0-140-53 kubenswrapper[2563]: I0706 23:28:33.206120    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4jvqw_b8a3215b-b208-420a-82f1-4907c79efac1/kube-rbac-proxy/0.log"