Jul 15 21:02:10.522291 ip-10-0-130-29 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 21:02:10.901966 ip-10-0-130-29 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 21:02:10.901966 ip-10-0-130-29 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 21:02:10.901966 ip-10-0-130-29 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 21:02:10.901966 ip-10-0-130-29 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 21:02:10.901966 ip-10-0-130-29 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 21:02:10.904432 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.904251    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 21:02:10.908620 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908597    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:02:10.908620 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908620    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:02:10.908620 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908624    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:02:10.908620 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908629    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:02:10.908620 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908632    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908635    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908638    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908641    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908644    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908647    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908649    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908652    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908655    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908657    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908660    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908662    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908665    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908669    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908672    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908674    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908677    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908680    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908682    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908687    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:02:10.908755 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908691    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908694    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908697    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908700    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908702    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908705    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908709    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908712    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908715    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908718    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908721    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908723    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908726    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908730    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908733    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908736    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908738    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908741    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908744    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:02:10.909216 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908747    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908749    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908752    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908754    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908757    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908760    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908763    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908772    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908775    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908778    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908780    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908783    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908785    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908788    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908791    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908795    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908798    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908801    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908804    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908807    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:02:10.909698 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908809    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908812    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908815    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908817    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908819    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908822    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908825    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908828    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908831    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908833    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908837    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908840    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908842    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908845    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908847    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908850    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908853    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908855    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908858    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908861    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:02:10.910180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908864    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908866    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908869    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908871    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908873    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908876    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908878    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908881    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908883    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908886    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908888    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908891    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908893    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908896    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908898    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.908901    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909616    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909622    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909625    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:02:10.910670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909629    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909632    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909635    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909637    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909640    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909643    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909645    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909648    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909650    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909653    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909655    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909658    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909660    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909663    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909666    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909668    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909671    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909673    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909676    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909678    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:02:10.911136 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909681    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909683    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909686    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909689    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909691    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909693    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909698    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909702    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909706    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909709    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909713    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909716    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909718    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909721    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909723    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909726    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909728    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909731    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909733    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:02:10.911602 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909736    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909738    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909741    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909744    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909746    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909748    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909752    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909755    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909758    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909760    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909763    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909766    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909768    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909772    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909775    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909778    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909780    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909783    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909785    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909788    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:02:10.912065 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909790    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909795    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909797    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909801    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909804    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909806    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909809    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909811    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909814    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909817    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909820    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909822    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909826    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909830    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909833    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909835    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909838    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909841    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909844    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:02:10.912585 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909846    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909849    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909852    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909854    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909856    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909860    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909863    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909865    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909868    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909870    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909873    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909876    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909879    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909881    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909883    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909886    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909889    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.909891    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910790    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910797    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:02:10.913054 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910801    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910804    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910807    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910811    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910815    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910818    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910821    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910824    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910827    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910830    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910833    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910835    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910838    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910840    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910843    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910845    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910848    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910851    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910854    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:02:10.913525 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910856    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910859    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910861    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910864    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910867    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910870    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910872    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910875    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910877    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910880    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910882    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910885    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910888    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910890    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910893    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910895    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910897    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910900    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910902    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910905    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910907    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:02:10.913976 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910910    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910912    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910915    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910917    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910920    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910922    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910925    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910927    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910930    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910933    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910936    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910939    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910941    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910944    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910947    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910949    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910952    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910968    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910971    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910974    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:02:10.914488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910976    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910979    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910982    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910984    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910987    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910989    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910992    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910995    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.910997    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911000    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911002    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911006    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911010    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911013    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911015    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911018    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911021    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911024    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911026    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911029    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:02:10.915001 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911032    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911038    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911040    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911043    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911046    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911049    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911053    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911056    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911059    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911062    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911065    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911068    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911071    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911074    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911076    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911079    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911081    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911146    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911152    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911158    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 21:02:10.915467 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911163    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911166    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911170    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911174    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911177    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911180    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911184    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911188    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911191    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911194    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911197    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911200    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911203    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911208    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911211    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911217    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911224    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911228    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911231    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911235    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911239    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911242    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911245    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 21:02:10.915923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911248    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911251    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911254    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911257    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911260    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911263    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911268    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911271    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911274    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911277    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911280    2580 flags.go:64] FLAG: --enable-server="true"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911283    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911287    2580 flags.go:64] FLAG: --event-burst="100"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911290    2580 flags.go:64] FLAG: --event-qps="50"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911293    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911296    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911299    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911302    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911305    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911308    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911311    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911314    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911318    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911321    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911325    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 21:02:10.916476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911329    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911332    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911335    2580 flags.go:64] FLAG: --feature-gates=""
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911338    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911341    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911344    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911347    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911350    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911353    2580 flags.go:64] FLAG: --help="false"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911356    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-130-29.ec2.internal"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911359    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911362    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911365    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911368    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911372    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911374    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911377    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911381    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911386    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911390    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911393    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911396    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911399    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911402    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 21:02:10.917119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911404    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911407    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911410    2580 flags.go:64] FLAG: --lock-file=""
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911413    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911415    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911420    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911426    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911429    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911433    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911436    2580 flags.go:64] FLAG: --logging-format="text"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911439    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911442    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911445    2580 flags.go:64] FLAG: --manifest-url=""
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911448    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911453    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911456    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911460    2580 flags.go:64] FLAG: --max-pods="110"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911463    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911466    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911468    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911471    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911474    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911477    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911480    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911487    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 21:02:10.917717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911490    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911493    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911496    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911499    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911505    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911507    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911510    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911513    2580 flags.go:64] FLAG: --port="10250"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911516    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911519    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01464f4a0864ea992"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911523    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911525    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911530    2580 flags.go:64] FLAG: --register-node="true"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911533    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911536    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911539    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911543    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911546    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911550    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911553    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911556    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911559    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911561    2580 flags.go:64] FLAG: --runonce="false"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911564    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911567    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 21:02:10.918703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911570    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911573    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911576    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911579    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911582    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911584    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911587    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911590    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911593    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911595    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911598    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911601    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911604    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911610    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911613    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911616    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911620    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911623    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911626    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911629    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911632    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911635    2580 flags.go:64] FLAG: --v="2"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911638    2580 flags.go:64] FLAG: --version="false"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911644    2580 flags.go:64] FLAG: --vmodule=""
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911648    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 21:02:10.919480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.911651    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911765    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911769    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911772    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911775    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911777    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911780    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911783    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911785    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911788    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911790    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911794    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911798    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911801    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911804    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911807    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911809    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911812    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911815    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:02:10.920092 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911817    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911820    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911823    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911825    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911828    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911830    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911833    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911836    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911839    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911841    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911843    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911847    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911850    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911853    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911855    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911858    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911860    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911863    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911865    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911868    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:02:10.920537 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911870    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911873    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911875    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911877    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911880    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911882    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911885    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911887    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911890    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911892    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911895    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911897    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911900    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911903    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911905    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911907    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911910    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911912    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911915    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911917    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:02:10.921116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911920    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911922    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911925    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911929    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911932    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911935    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911937    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911941    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911943    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911946    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911948    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911950    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911968    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911971    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911974    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911977    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911980    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911982    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911985    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911987    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:02:10.921628 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911990    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911992    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911995    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.911999    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912003    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912006    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912009    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912013    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912016    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912019    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912022    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912024    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912027    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912030    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912033    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912037    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912040    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912042    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912045    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:02:10.922116 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912048    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.912051    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.912057    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.919894    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.919909    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.919928    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.919935    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920009    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920015    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920018    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920021    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920024    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920027    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920029    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:02:10.922576 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920032    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920035    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920037    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920040    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920042    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920045    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920048    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920050    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920053    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920056    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920059    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920062    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920064    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920067    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920070    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920072    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920075    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920078    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920080    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920083    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:02:10.922918 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920087    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920090    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920094    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920098    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920102    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920106    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920109    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920113    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920116    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920119    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920121    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920124    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920127    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920129    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920132    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920134    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920137    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920139    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920142    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:02:10.923409 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920145    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920148    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920151    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920153    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920156    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920159    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920161    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920164    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920166    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920169    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920171    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920174    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920177    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920179    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920182    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920185    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920188    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920191    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920193    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:02:10.923907 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920196    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920199    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920201    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920204    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920207    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920210    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920212    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920215    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920218    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920220    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920223    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920226    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920229    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920231    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920234    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920237    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920240    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920242    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920245    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920247    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:02:10.924398 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920250    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920252    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920255    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920258    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920260    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920263    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920265    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920268    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920271    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920275    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920278    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920281    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920284    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920287    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:02:10.924860 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.920292    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920401    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920405    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920409    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920412    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920414    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920417    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920420    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920423    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920425    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920429    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920433    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920436    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920439    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920442    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920445    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920447    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920450    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920453    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920455    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 21:02:10.925222 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920458    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920460    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920463    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920465    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920468    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920471    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920474    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920477    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920480    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920482    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920485    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920488    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920491    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920495    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920498    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920501    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920504    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920506    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920509    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 21:02:10.925693 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920511    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920514    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920516    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920519    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920522    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920524    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920527    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920530    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920533    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920535    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920538    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920540    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920543    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920546    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920548    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920551    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920554    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920557    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920560    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920562    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 21:02:10.926159 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920565    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920567    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920570    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920572    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920575    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920577    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920580    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920582    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920585    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920587    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920590    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920592    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920594    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920597    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920599    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920602    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920604    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920606    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920609    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920612    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 21:02:10.926632 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920615    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920618    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920620    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920622    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920625    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920627    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920630    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920632    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920635    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920638    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920641    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920643    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920645    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920648    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920651    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920653    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920656    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920659    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920663    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 21:02:10.927175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920665    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 21:02:10.927626 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:10.920668    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 21:02:10.927626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.920673    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 21:02:10.927626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.920771    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 21:02:10.927626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.923529    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 21:02:10.927626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.924671    2580 server.go:1019] "Starting client certificate rotation"
Jul 15 21:02:10.927626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.924763    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 21:02:10.927626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.924795    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 21:02:10.927626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.926976    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 21:02:10.931470 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.931453    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 21:02:10.939542 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.939523    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 15 21:02:10.942293 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.942280    2580 log.go:25] "Validated CRI v1 image API"
Jul 15 21:02:10.943856 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.943844    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 21:02:10.947209 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.947192    2580 fs.go:135] Filesystem UUIDs: map[1387db19-c416-4200-944d-a042720c836f:/dev/nvme0n1p4 1dece3fc-f750-49ca-b161-287081a8af05:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 15 21:02:10.947262 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.947210    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 21:02:10.953442 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.953321    2580 manager.go:216] Machine: {Timestamp:2026-07-15 21:02:10.951784992 +0000 UTC m=+0.308703579 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3089727 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec277893ec3abe360de1ebdc2d70c46e SystemUUID:ec277893-ec3a-be36-0de1-ebdc2d70c46e BootID:642e6d52-5b42-47b8-9a6b-955d2e4835f3 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:08:aa:3f:19:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:08:aa:3f:19:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:74:74:17:40:65 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 21:02:10.953442 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.953428    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 21:02:10.953583 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.953511    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 21:02:10.956174 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.956155    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 21:02:10.956396 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.956371    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 21:02:10.956530 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.956402    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-29.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 21:02:10.957095 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.957086    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 21:02:10.957133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.957097    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 21:02:10.957133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.957105    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 21:02:10.957722 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.957711    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 21:02:10.958858 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.958847    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 21:02:10.961509 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.961497    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 21:02:10.961638 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.961628    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 21:02:10.961674 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.961658    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 21:02:10.961674 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.961670    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 21:02:10.961728 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.961684    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 21:02:10.961728 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.961694    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 15 21:02:10.961728 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.961703    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 21:02:10.962072 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.962060    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 21:02:10.962132 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.962081    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 21:02:10.964097 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.964077    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 21:02:10.965847 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.965829    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 21:02:10.965925 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.965869    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 21:02:10.966095 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966082    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 21:02:10.966160 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966100    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 21:02:10.966160 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966110    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 21:02:10.966160 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966118    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 21:02:10.966160 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966129    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 21:02:10.966160 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966138    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 21:02:10.966485 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966475    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 21:02:10.966536 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966488    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 21:02:10.966536 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966499    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 21:02:10.966536 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966508    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 21:02:10.966536 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966521    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 21:02:10.966536 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.966534    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 21:02:10.967243 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.967232    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 21:02:10.967292 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.967245    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 21:02:10.970475 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.970452    2580 server.go:1268] "Started kubelet"
Jul 15 21:02:10.970620 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.970569    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 21:02:10.970679 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.970632    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 21:02:10.970725 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.970710    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 21:02:10.971676 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.971656    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 21:02:10.971973 ip-10-0-130-29 systemd[1]: Started Kubernetes Kubelet.
Jul 15 21:02:10.973114 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.973068    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 21:02:10.973757 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.973735    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-29.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 21:02:10.973838 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:10.973742    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 21:02:10.973838 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:10.973773    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-29.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 21:02:10.975772 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:10.975752    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 21:02:10.976234 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:10.975371    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-29.ec2.internal.18c2925d1abdee74  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-29.ec2.internal,UID:ip-10-0-130-29.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-29.ec2.internal,},FirstTimestamp:2026-07-15 21:02:10.970406516 +0000 UTC m=+0.327325103,LastTimestamp:2026-07-15 21:02:10.970406516 +0000 UTC m=+0.327325103,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-29.ec2.internal,}"
Jul 15 21:02:10.978637 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.978618    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 21:02:10.978637 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.978629    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 21:02:10.978766 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:10.978722    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:10.979164 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.979131    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 21:02:10.979243 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.979188    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 21:02:10.979243 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.979212    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 21:02:10.979378 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.979346    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 21:02:10.979476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.979393    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 21:02:10.980306 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.980293    2580 factory.go:153] Registering CRI-O factory
Jul 15 21:02:10.980386 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.980309    2580 factory.go:223] Registration of the crio container factory successfully
Jul 15 21:02:10.980386 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.980367    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 21:02:10.980386 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.980374    2580 factory.go:55] Registering systemd factory
Jul 15 21:02:10.980386 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.980379    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 15 21:02:10.980506 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.980395    2580 factory.go:103] Registering Raw factory
Jul 15 21:02:10.980506 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.980407    2580 manager.go:1196] Started watching for new ooms in manager
Jul 15 21:02:10.980923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.980909    2580 manager.go:349] Starting recovery of all containers
Jul 15 21:02:10.985218 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:10.985157    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-29.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 21:02:10.985688 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:10.985666    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 21:02:10.992263 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.992244    2580 manager.go:354] Recovery completed
Jul 15 21:02:10.997500 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.997488    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:02:10.999658 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.999639    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:02:10.999727 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.999676    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:02:10.999727 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:10.999690    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:02:11.000355 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.000342    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 21:02:11.000412 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.000355    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 21:02:11.000412 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.000371    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 21:02:11.000504 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.000415    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:02:11.001184 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.001168    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:02:11.001282 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.001197    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:02:11.001282 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.001213    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:02:11.001509 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.001497    2580 policy_none.go:49] "None policy: Start"
Jul 15 21:02:11.001553 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.001514    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 21:02:11.001553 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.001524    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 21:02:11.002082 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.002060    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6mq9j"
Jul 15 21:02:11.002303 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.002239    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-29.ec2.internal.18c2925d1c7c4846  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-29.ec2.internal,UID:ip-10-0-130-29.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-29.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-29.ec2.internal,},FirstTimestamp:2026-07-15 21:02:10.999658566 +0000 UTC m=+0.356577162,LastTimestamp:2026-07-15 21:02:10.999658566 +0000 UTC m=+0.356577162,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-29.ec2.internal,}"
Jul 15 21:02:11.002526 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.002514    2580 policy_none.go:47] "Start"
Jul 15 21:02:11.010149 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.010131    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6mq9j"
Jul 15 21:02:11.011688 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.011624    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-29.ec2.internal.18c2925d1c7ca7ca  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-29.ec2.internal,UID:ip-10-0-130-29.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-29.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-29.ec2.internal,},FirstTimestamp:2026-07-15 21:02:10.999683018 +0000 UTC m=+0.356601605,LastTimestamp:2026-07-15 21:02:10.999683018 +0000 UTC m=+0.356601605,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-29.ec2.internal,}"
Jul 15 21:02:11.013482 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.013428    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-29.ec2.internal.18c2925d1c7cd42a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-29.ec2.internal,UID:ip-10-0-130-29.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-29.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-29.ec2.internal,},FirstTimestamp:2026-07-15 21:02:10.999694378 +0000 UTC m=+0.356612965,LastTimestamp:2026-07-15 21:02:10.999694378 +0000 UTC m=+0.356612965,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-29.ec2.internal,}"
Jul 15 21:02:11.040661 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.040514    2580 manager.go:337] "Starting Device Plugin manager"
Jul 15 21:02:11.040783 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.040696    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 21:02:11.040783 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.040712    2580 server.go:87] "Starting device plugin registration server"
Jul 15 21:02:11.040970 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.040948    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 21:02:11.041021 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.040979    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 21:02:11.041090 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.041074    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 21:02:11.041183 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.041163    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 21:02:11.041183 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.041176    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 21:02:11.041660 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.041640    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 21:02:11.041724 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.041686    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.126537 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.126501    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 21:02:11.127749 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.127733    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 21:02:11.127812 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.127771    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 21:02:11.127812 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.127799    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 21:02:11.127918 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.127835    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 21:02:11.130611 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.130595    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:02:11.142005 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.141989    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:02:11.143185 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.143169    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:02:11.143263 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.143200    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:02:11.143263 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.143214    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:02:11.143263 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.143248    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.150564 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.150541    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.150650 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.150573    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-29.ec2.internal\": node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.184782 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.184716    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.228508 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.228480    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"]
Jul 15 21:02:11.228640 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.228551    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:02:11.230178 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.230158    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:02:11.230279 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.230187    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:02:11.230279 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.230199    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:02:11.231433 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.231421    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:02:11.231537 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.231524    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.231583 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.231551    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:02:11.232205 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.232188    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:02:11.232303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.232215    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:02:11.232303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.232221    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:02:11.232303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.232228    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:02:11.232303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.232243    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:02:11.232303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.232267    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:02:11.233266 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.233252    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.233323 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.233282    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 21:02:11.233941 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.233918    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 21:02:11.234032 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.233950    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 21:02:11.234032 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.233991    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeHasSufficientPID"
Jul 15 21:02:11.254548 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.254529    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-29.ec2.internal\" not found" node="ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.258542 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.258526    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-29.ec2.internal\" not found" node="ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.281677 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.281649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.281747 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.281681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.281747 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.281699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46ea3c93a970f7700a2a3849415d7a22-config\") pod \"kube-apiserver-proxy-ip-10-0-130-29.ec2.internal\" (UID: \"46ea3c93a970f7700a2a3849415d7a22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.285748 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.285734    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.382019 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.381989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.382123 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.382025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.382123 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.382042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46ea3c93a970f7700a2a3849415d7a22-config\") pod \"kube-apiserver-proxy-ip-10-0-130-29.ec2.internal\" (UID: \"46ea3c93a970f7700a2a3849415d7a22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.382123 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.382080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46ea3c93a970f7700a2a3849415d7a22-config\") pod \"kube-apiserver-proxy-ip-10-0-130-29.ec2.internal\" (UID: \"46ea3c93a970f7700a2a3849415d7a22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.382123 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.382088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.382123 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.382097    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e866a2a89b76039c681f32045b5640c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal\" (UID: \"4e866a2a89b76039c681f32045b5640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.386083 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.386066    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.486854 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.486792    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.557037 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.556995    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.561815 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.561796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.587047 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.587014    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.687565 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.687521    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.788083 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.788001    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.888634 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.888597    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-29.ec2.internal\" not found"
Jul 15 21:02:11.897413 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.897389    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:02:11.924639 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.924612    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 21:02:11.925141 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.924742    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 21:02:11.925141 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.924808    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 21:02:11.961875 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.961840    2580 apiserver.go:52] "Watching apiserver"
Jul 15 21:02:11.977504 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.977482    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 21:02:11.978706 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.978677    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6ldsz","kube-system/konnectivity-agent-nhsnr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg","openshift-cluster-node-tuning-operator/tuned-qncr2","openshift-multus/multus-additional-cni-plugins-dsjsm","openshift-multus/multus-wvkjb","openshift-network-diagnostics/network-check-target-kmgwk","openshift-ovn-kubernetes/ovnkube-node-gckpk","openshift-dns/node-resolver-599sr","openshift-image-registry/node-ca-lc5h4","openshift-multus/network-metrics-daemon-xr98b","openshift-network-operator/iptables-alerter-cbg28"]
Jul 15 21:02:11.978706 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.978701    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 21:02:11.979112 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.979092    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.980006 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.979989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:11.980107 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.980053    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:11.981027 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.981013    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:11.982257 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.982239    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:11.983106 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.983092    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nd9wk\""
Jul 15 21:02:11.983437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.983422    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 21:02:11.983545 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.983455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:11.983851 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.983838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 21:02:11.984143 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.984124    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 21:02:11.984143 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.984134    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 21:02:11.984423 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.984404    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 21:02:11.984570 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.984552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:11.984671 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.984578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e184053-7574-434b-8873-db1b343c9589-kubelet-config\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:11.984671 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.984594    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e184053-7574-434b-8873-db1b343c9589-dbus\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:11.984763 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.984740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:11.985017 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.984901    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d7vs5\""
Jul 15 21:02:11.986001 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.985983    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:11.987253 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.987232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:11.987342 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.987310    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:11.989018 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.989001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:11.989650 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.989636    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 21:02:11.990103 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.990081    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 21:02:11.990196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.990089    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 21:02:11.990660 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.990645    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:11.991775 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.991760    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:11.991988 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.991943    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 21:02:11.992347 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.992327    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2gwfp\""
Jul 15 21:02:11.992474 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.992455    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 21:02:11.992474 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.992463    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 21:02:11.992626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.992548    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 21:02:11.992626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.992464    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 21:02:11.992626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.992586    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 21:02:11.993020 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993002    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:11.993121 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993077    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 21:02:11.993121 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:11.993071    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:11.993327 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993306    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-v65k9\""
Jul 15 21:02:11.993455 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993410    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-grr45\""
Jul 15 21:02:11.993516 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993484    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 21:02:11.993516 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993492    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 21:02:11.993606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993590    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 21:02:11.993650 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993608    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 21:02:11.993689 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993650    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fpkdl\""
Jul 15 21:02:11.993883 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993868    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 21:02:11.994042 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.993984    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 21:02:11.994042 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.994012    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w5dmv\""
Jul 15 21:02:11.994448 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.994432    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 21:02:11.994731 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.994716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:11.995090 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.995075    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kk7mp\""
Jul 15 21:02:11.995458 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.995441    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 21:02:11.995605 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.995524    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"
Jul 15 21:02:11.996479 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.996347    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 21:02:11.996479 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.996357    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 21:02:11.998239 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:11.997976    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal"]
Jul 15 21:02:12.000048 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.000027    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 21:02:12.000153 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.000055    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 21:02:12.001408 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.001388    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 21:02:12.001578 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.001564    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 21:02:12.002355 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.002340    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cq9vw\""
Jul 15 21:02:12.007534 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.007515    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal"]
Jul 15 21:02:12.008090 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.008074    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 21:02:12.012776 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.012740    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 20:57:11 +0000 UTC" deadline="2028-02-29 20:03:18.471757504 +0000 UTC"
Jul 15 21:02:12.012850 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.012776    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14255h1m6.458984785s"
Jul 15 21:02:12.026166 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.026144    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8w9g"
Jul 15 21:02:12.040772 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.040707    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8w9g"
Jul 15 21:02:12.080088 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.080069    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 21:02:12.085066 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96932186-bd45-4160-b3e7-70509123eff9-tmp\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.085159 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-socket-dir-parent\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.085159 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a5ff2fdc-8161-49f8-add9-8393e61dd94f-env-overrides\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.085159 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-os-release\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.085159 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1899eb2-d219-4554-910b-e0e59f2b22c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.085336 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e-serviceca\") pod \"node-ca-lc5h4\" (UID: \"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e\") " pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.085336 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/330f78ca-c79c-409c-9185-3f188ddb8086-host-slash\") pod \"iptables-alerter-cbg28\" (UID: \"330f78ca-c79c-409c-9185-3f188ddb8086\") " pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.085336 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-etc-kubernetes\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.085336 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-system-cni-dir\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.085336 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.085336 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-device-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.085336 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-run-k8s-cni-cncf-io\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-run-multus-certs\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-kubelet\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-node-log\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-sysctl-conf\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-registration-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-cnibin\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-var-lib-cni-bin\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.085531    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a5ff2fdc-8161-49f8-add9-8393e61dd94f-ovnkube-config\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e184053-7574-434b-8873-db1b343c9589-dbus\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7fjq\" (UniqueName: \"kubernetes.io/projected/b1899eb2-d219-4554-910b-e0e59f2b22c2-kube-api-access-n7fjq\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e-host\") pod \"node-ca-lc5h4\" (UID: \"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e\") " pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5sjj\" (UniqueName: \"kubernetes.io/projected/e3220222-dfc1-4a57-a205-809e0fba860e-kube-api-access-g5sjj\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:12.085653 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.085634    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret podName:4e184053-7574-434b-8873-db1b343c9589 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:12.58560763 +0000 UTC m=+1.942526216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret") pod "global-pull-secret-syncer-6ldsz" (UID: "4e184053-7574-434b-8873-db1b343c9589") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-socket-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-cni-dir\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-hostroot\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-run-netns\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e184053-7574-434b-8873-db1b343c9589-dbus\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-slash\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-run-systemd\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-cni-netd\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a5ff2fdc-8161-49f8-add9-8393e61dd94f-ovnkube-script-lib\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-sysconfig\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-cnibin\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/855e91fe-3c9d-4f36-a073-09eecd10ed4c-tmp-dir\") pod \"node-resolver-599sr\" (UID: \"855e91fe-3c9d-4f36-a073-09eecd10ed4c\") " pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085896    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/330f78ca-c79c-409c-9185-3f188ddb8086-iptables-alerter-script\") pod \"iptables-alerter-cbg28\" (UID: \"330f78ca-c79c-409c-9185-3f188ddb8086\") " pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-system-cni-dir\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-var-lib-openvswitch\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-etc-openvswitch\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.085985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-log-socket\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-lib-modules\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5zqz\" (UniqueName: \"kubernetes.io/projected/94b810f1-fbc9-418f-811f-186122b7027a-kube-api-access-r5zqz\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-cni-binary-copy\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086087    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-run-netns\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-run-openvswitch\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcbsf\" (UniqueName: \"kubernetes.io/projected/a5ff2fdc-8161-49f8-add9-8393e61dd94f-kube-api-access-pcbsf\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr798\" (UniqueName: \"kubernetes.io/projected/0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e-kube-api-access-sr798\") pod \"node-ca-lc5h4\" (UID: \"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e\") " pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-os-release\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/855e91fe-3c9d-4f36-a073-09eecd10ed4c-hosts-file\") pod \"node-resolver-599sr\" (UID: \"855e91fe-3c9d-4f36-a073-09eecd10ed4c\") " pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3517b0c1-318b-457b-b272-efc0e289dbbc-konnectivity-ca\") pod \"konnectivity-agent-nhsnr\" (UID: \"3517b0c1-318b-457b-b272-efc0e289dbbc\") " pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdnkx\" (UniqueName: \"kubernetes.io/projected/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-kube-api-access-vdnkx\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l5d4\" (UniqueName: \"kubernetes.io/projected/855e91fe-3c9d-4f36-a073-09eecd10ed4c-kube-api-access-6l5d4\") pod \"node-resolver-599sr\" (UID: \"855e91fe-3c9d-4f36-a073-09eecd10ed4c\") " pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3517b0c1-318b-457b-b272-efc0e289dbbc-agent-certs\") pod \"konnectivity-agent-nhsnr\" (UID: \"3517b0c1-318b-457b-b272-efc0e289dbbc\") " pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-daemon-config\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.086892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-systemd-units\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5ff2fdc-8161-49f8-add9-8393e61dd94f-ovn-node-metrics-cert\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e184053-7574-434b-8873-db1b343c9589-kubelet-config\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-var-lib-kubelet\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-etc-selinux\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086471    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e184053-7574-434b-8873-db1b343c9589-kubelet-config\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-conf-dir\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-run-ovn\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-cni-bin\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-kubernetes\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-run\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086580    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-sys\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-var-lib-cni-multus\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-var-lib-kubelet\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-run-ovn-kubernetes\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.087407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086668    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-systemd\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1899eb2-d219-4554-910b-e0e59f2b22c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgdbs\" (UniqueName: \"kubernetes.io/projected/330f78ca-c79c-409c-9185-3f188ddb8086-kube-api-access-fgdbs\") pod \"iptables-alerter-cbg28\" (UID: \"330f78ca-c79c-409c-9185-3f188ddb8086\") " pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-modprobe-d\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-host\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1899eb2-d219-4554-910b-e0e59f2b22c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-sysctl-d\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/96932186-bd45-4160-b3e7-70509123eff9-etc-tuned\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkll8\" (UniqueName: \"kubernetes.io/projected/96932186-bd45-4160-b3e7-70509123eff9-kube-api-access-nkll8\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.087834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.086867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-sys-fs\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.111301 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.111269    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ea3c93a970f7700a2a3849415d7a22.slice/crio-d4760fd1ce59ff6c06e8ce7f6e4389a61f3e178b62fec87da5f578d342534f9d WatchSource:0}: Error finding container d4760fd1ce59ff6c06e8ce7f6e4389a61f3e178b62fec87da5f578d342534f9d: Status 404 returned error can't find the container with id d4760fd1ce59ff6c06e8ce7f6e4389a61f3e178b62fec87da5f578d342534f9d
Jul 15 21:02:12.111700 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.111684    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e866a2a89b76039c681f32045b5640c.slice/crio-4904f02a135955ddcd0a4ff614b84d658ab887258ec71dd518d4276de32ec590 WatchSource:0}: Error finding container 4904f02a135955ddcd0a4ff614b84d658ab887258ec71dd518d4276de32ec590: Status 404 returned error can't find the container with id 4904f02a135955ddcd0a4ff614b84d658ab887258ec71dd518d4276de32ec590
Jul 15 21:02:12.115862 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.115840    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:02:12.130844 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.130803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal" event={"ID":"4e866a2a89b76039c681f32045b5640c","Type":"ContainerStarted","Data":"4904f02a135955ddcd0a4ff614b84d658ab887258ec71dd518d4276de32ec590"}
Jul 15 21:02:12.131907 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.131881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal" event={"ID":"46ea3c93a970f7700a2a3849415d7a22","Type":"ContainerStarted","Data":"d4760fd1ce59ff6c06e8ce7f6e4389a61f3e178b62fec87da5f578d342534f9d"}
Jul 15 21:02:12.187383 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187351    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-modprobe-d\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.187383 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-host\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1899eb2-d219-4554-910b-e0e59f2b22c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-sysctl-d\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/96932186-bd45-4160-b3e7-70509123eff9-etc-tuned\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkll8\" (UniqueName: \"kubernetes.io/projected/96932186-bd45-4160-b3e7-70509123eff9-kube-api-access-nkll8\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-sys-fs\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187482    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-host\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-sys-fs\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96932186-bd45-4160-b3e7-70509123eff9-tmp\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-modprobe-d\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-socket-dir-parent\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-socket-dir-parent\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.187613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a5ff2fdc-8161-49f8-add9-8393e61dd94f-env-overrides\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-os-release\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1899eb2-d219-4554-910b-e0e59f2b22c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e-serviceca\") pod \"node-ca-lc5h4\" (UID: \"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e\") " pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/330f78ca-c79c-409c-9185-3f188ddb8086-host-slash\") pod \"iptables-alerter-cbg28\" (UID: \"330f78ca-c79c-409c-9185-3f188ddb8086\") " pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-etc-kubernetes\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187750    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-os-release\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-system-cni-dir\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187785    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/330f78ca-c79c-409c-9185-3f188ddb8086-host-slash\") pod \"iptables-alerter-cbg28\" (UID: \"330f78ca-c79c-409c-9185-3f188ddb8086\") " pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-sysctl-d\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187792    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-system-cni-dir\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187813    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-device-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-run-k8s-cni-cncf-io\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-run-multus-certs\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-kubelet\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.188196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-node-log\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-sysctl-conf\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-registration-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187995    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-cnibin\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-run-multus-certs\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-var-lib-cni-bin\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-run-k8s-cni-cncf-io\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a5ff2fdc-8161-49f8-add9-8393e61dd94f-ovnkube-config\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-node-log\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7fjq\" (UniqueName: \"kubernetes.io/projected/b1899eb2-d219-4554-910b-e0e59f2b22c2-kube-api-access-n7fjq\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1899eb2-d219-4554-910b-e0e59f2b22c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e-host\") pod \"node-ca-lc5h4\" (UID: \"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e\") " pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-var-lib-cni-bin\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5sjj\" (UniqueName: \"kubernetes.io/projected/e3220222-dfc1-4a57-a205-809e0fba860e-kube-api-access-g5sjj\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e-serviceca\") pod \"node-ca-lc5h4\" (UID: \"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e\") " pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.187851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-etc-kubernetes\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-sysctl-conf\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.188832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188217    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e-host\") pod \"node-ca-lc5h4\" (UID: \"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e\") " pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188217    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-device-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-kubelet\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-socket-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-cnibin\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-cni-dir\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-registration-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-hostroot\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1899eb2-d219-4554-910b-e0e59f2b22c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-run-netns\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-slash\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-run-systemd\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-cni-dir\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-socket-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-cni-netd\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-run-netns\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a5ff2fdc-8161-49f8-add9-8393e61dd94f-ovnkube-script-lib\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188417    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-hostroot\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.189685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188417    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-run-systemd\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-sysconfig\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-cni-netd\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188457    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-slash\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-cnibin\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/855e91fe-3c9d-4f36-a073-09eecd10ed4c-tmp-dir\") pod \"node-resolver-599sr\" (UID: \"855e91fe-3c9d-4f36-a073-09eecd10ed4c\") " pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-sysconfig\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/330f78ca-c79c-409c-9185-3f188ddb8086-iptables-alerter-script\") pod \"iptables-alerter-cbg28\" (UID: \"330f78ca-c79c-409c-9185-3f188ddb8086\") " pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188530    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1899eb2-d219-4554-910b-e0e59f2b22c2-cnibin\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-system-cni-dir\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-var-lib-openvswitch\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a5ff2fdc-8161-49f8-add9-8393e61dd94f-ovnkube-config\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-etc-openvswitch\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-log-socket\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a5ff2fdc-8161-49f8-add9-8393e61dd94f-env-overrides\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-lib-modules\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5zqz\" (UniqueName: \"kubernetes.io/projected/94b810f1-fbc9-418f-811f-186122b7027a-kube-api-access-r5zqz\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.190582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-cni-binary-copy\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188682    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-etc-openvswitch\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-var-lib-openvswitch\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188735    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-log-socket\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-run-netns\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-run-openvswitch\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcbsf\" (UniqueName: \"kubernetes.io/projected/a5ff2fdc-8161-49f8-add9-8393e61dd94f-kube-api-access-pcbsf\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/855e91fe-3c9d-4f36-a073-09eecd10ed4c-tmp-dir\") pod \"node-resolver-599sr\" (UID: \"855e91fe-3c9d-4f36-a073-09eecd10ed4c\") " pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-system-cni-dir\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr798\" (UniqueName: \"kubernetes.io/projected/0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e-kube-api-access-sr798\") pod \"node-ca-lc5h4\" (UID: \"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e\") " pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-os-release\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188891    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-run-netns\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/855e91fe-3c9d-4f36-a073-09eecd10ed4c-hosts-file\") pod \"node-resolver-599sr\" (UID: \"855e91fe-3c9d-4f36-a073-09eecd10ed4c\") " pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a5ff2fdc-8161-49f8-add9-8393e61dd94f-ovnkube-script-lib\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3517b0c1-318b-457b-b272-efc0e289dbbc-konnectivity-ca\") pod \"konnectivity-agent-nhsnr\" (UID: \"3517b0c1-318b-457b-b272-efc0e289dbbc\") " pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdnkx\" (UniqueName: \"kubernetes.io/projected/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-kube-api-access-vdnkx\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l5d4\" (UniqueName: \"kubernetes.io/projected/855e91fe-3c9d-4f36-a073-09eecd10ed4c-kube-api-access-6l5d4\") pod \"node-resolver-599sr\" (UID: \"855e91fe-3c9d-4f36-a073-09eecd10ed4c\") " pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.191384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-os-release\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3517b0c1-318b-457b-b272-efc0e289dbbc-agent-certs\") pod \"konnectivity-agent-nhsnr\" (UID: \"3517b0c1-318b-457b-b272-efc0e289dbbc\") " pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-daemon-config\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-systemd-units\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5ff2fdc-8161-49f8-add9-8393e61dd94f-ovn-node-metrics-cert\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-var-lib-kubelet\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/330f78ca-c79c-409c-9185-3f188ddb8086-iptables-alerter-script\") pod \"iptables-alerter-cbg28\" (UID: \"330f78ca-c79c-409c-9185-3f188ddb8086\") " pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-etc-selinux\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188932    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-run-openvswitch\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-conf-dir\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-run-ovn\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/855e91fe-3c9d-4f36-a073-09eecd10ed4c-hosts-file\") pod \"node-resolver-599sr\" (UID: \"855e91fe-3c9d-4f36-a073-09eecd10ed4c\") " pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.188829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-lib-modules\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-etc-selinux\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189283    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-run-ovn\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-cni-bin\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.191933 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-kubernetes\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-run\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-sys\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-var-lib-cni-multus\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-var-lib-kubelet\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-run-ovn-kubernetes\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-systemd\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1899eb2-d219-4554-910b-e0e59f2b22c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-sys\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgdbs\" (UniqueName: \"kubernetes.io/projected/330f78ca-c79c-409c-9185-3f188ddb8086-kube-api-access-fgdbs\") pod \"iptables-alerter-cbg28\" (UID: \"330f78ca-c79c-409c-9185-3f188ddb8086\") " pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-kubernetes\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189605    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-cni-bin\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-conf-dir\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-run\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.189726    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:12.192420 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.189789    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs podName:e3220222-dfc1-4a57-a205-809e0fba860e nodeName:}" failed. No retries permitted until 2026-07-15 21:02:12.689773124 +0000 UTC m=+2.046691718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs") pod "network-metrics-daemon-xr98b" (UID: "e3220222-dfc1-4a57-a205-809e0fba860e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94b810f1-fbc9-418f-811f-186122b7027a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.189725    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.190006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-systemd-units\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.190007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-var-lib-kubelet\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.190036    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3517b0c1-318b-457b-b272-efc0e289dbbc-konnectivity-ca\") pod \"konnectivity-agent-nhsnr\" (UID: \"3517b0c1-318b-457b-b272-efc0e289dbbc\") " pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.190042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-host-var-lib-cni-multus\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.190091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5ff2fdc-8161-49f8-add9-8393e61dd94f-host-run-ovn-kubernetes\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.190101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-etc-systemd\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.190114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1899eb2-d219-4554-910b-e0e59f2b22c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.190152    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96932186-bd45-4160-b3e7-70509123eff9-var-lib-kubelet\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.190611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-multus-daemon-config\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.191045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-cni-binary-copy\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.191561    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/96932186-bd45-4160-b3e7-70509123eff9-etc-tuned\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.191697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96932186-bd45-4160-b3e7-70509123eff9-tmp\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.192120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5ff2fdc-8161-49f8-add9-8393e61dd94f-ovn-node-metrics-cert\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.192898 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.192174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3517b0c1-318b-457b-b272-efc0e289dbbc-agent-certs\") pod \"konnectivity-agent-nhsnr\" (UID: \"3517b0c1-318b-457b-b272-efc0e289dbbc\") " pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:12.199161 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.199140    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5zqz\" (UniqueName: \"kubernetes.io/projected/94b810f1-fbc9-418f-811f-186122b7027a-kube-api-access-r5zqz\") pod \"aws-ebs-csi-driver-node-rlmwg\" (UID: \"94b810f1-fbc9-418f-811f-186122b7027a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.199341 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.199320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l5d4\" (UniqueName: \"kubernetes.io/projected/855e91fe-3c9d-4f36-a073-09eecd10ed4c-kube-api-access-6l5d4\") pod \"node-resolver-599sr\" (UID: \"855e91fe-3c9d-4f36-a073-09eecd10ed4c\") " pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.199341 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.199334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr798\" (UniqueName: \"kubernetes.io/projected/0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e-kube-api-access-sr798\") pod \"node-ca-lc5h4\" (UID: \"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e\") " pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.199458 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.199372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgdbs\" (UniqueName: \"kubernetes.io/projected/330f78ca-c79c-409c-9185-3f188ddb8086-kube-api-access-fgdbs\") pod \"iptables-alerter-cbg28\" (UID: \"330f78ca-c79c-409c-9185-3f188ddb8086\") " pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.199602 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.199586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcbsf\" (UniqueName: \"kubernetes.io/projected/a5ff2fdc-8161-49f8-add9-8393e61dd94f-kube-api-access-pcbsf\") pod \"ovnkube-node-gckpk\" (UID: \"a5ff2fdc-8161-49f8-add9-8393e61dd94f\") " pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.199751 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.199735    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkll8\" (UniqueName: \"kubernetes.io/projected/96932186-bd45-4160-b3e7-70509123eff9-kube-api-access-nkll8\") pod \"tuned-qncr2\" (UID: \"96932186-bd45-4160-b3e7-70509123eff9\") " pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.200137 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.200107    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5sjj\" (UniqueName: \"kubernetes.io/projected/e3220222-dfc1-4a57-a205-809e0fba860e-kube-api-access-g5sjj\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:12.201892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.201875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7fjq\" (UniqueName: \"kubernetes.io/projected/b1899eb2-d219-4554-910b-e0e59f2b22c2-kube-api-access-n7fjq\") pod \"multus-additional-cni-plugins-dsjsm\" (UID: \"b1899eb2-d219-4554-910b-e0e59f2b22c2\") " pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.202626 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.202610    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:02:12.202688 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.202630    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:02:12.202688 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.202641    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nm9j9 for pod openshift-network-diagnostics/network-check-target-kmgwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:12.202765 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.202702    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9 podName:31bad667-66ee-41eb-9643-a72135c77989 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:12.702686271 +0000 UTC m=+2.059604857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nm9j9" (UniqueName: "kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9") pod "network-check-target-kmgwk" (UID: "31bad667-66ee-41eb-9643-a72135c77989") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:12.204402 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.204387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdnkx\" (UniqueName: \"kubernetes.io/projected/e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3-kube-api-access-vdnkx\") pod \"multus-wvkjb\" (UID: \"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3\") " pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.216379 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.216358    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:02:12.306738 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.306648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:12.316076 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.316051    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3517b0c1_318b_457b_b272_efc0e289dbbc.slice/crio-c3eadad065b3bf5df4a20514a61980754a9f8c98ee526646c420c65ca8aabb65 WatchSource:0}: Error finding container c3eadad065b3bf5df4a20514a61980754a9f8c98ee526646c420c65ca8aabb65: Status 404 returned error can't find the container with id c3eadad065b3bf5df4a20514a61980754a9f8c98ee526646c420c65ca8aabb65
Jul 15 21:02:12.316607 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.316587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg"
Jul 15 21:02:12.323886 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.323864    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b810f1_fbc9_418f_811f_186122b7027a.slice/crio-fbb62bc1c2b5488768b086c4019ae1d5f4450d2021722780e7292ab528dd8ca4 WatchSource:0}: Error finding container fbb62bc1c2b5488768b086c4019ae1d5f4450d2021722780e7292ab528dd8ca4: Status 404 returned error can't find the container with id fbb62bc1c2b5488768b086c4019ae1d5f4450d2021722780e7292ab528dd8ca4
Jul 15 21:02:12.329882 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.329858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qncr2"
Jul 15 21:02:12.336901 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.336876    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96932186_bd45_4160_b3e7_70509123eff9.slice/crio-d627d60d3c805f37be7b4fd563d12c8cce94914b42d80babd1c34b61550e5347 WatchSource:0}: Error finding container d627d60d3c805f37be7b4fd563d12c8cce94914b42d80babd1c34b61550e5347: Status 404 returned error can't find the container with id d627d60d3c805f37be7b4fd563d12c8cce94914b42d80babd1c34b61550e5347
Jul 15 21:02:12.343529 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.343509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dsjsm"
Jul 15 21:02:12.352382 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.352359    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1899eb2_d219_4554_910b_e0e59f2b22c2.slice/crio-a984c31b971b12da08a9d7e867a73f5cf3528455a2e51208739c17712450190d WatchSource:0}: Error finding container a984c31b971b12da08a9d7e867a73f5cf3528455a2e51208739c17712450190d: Status 404 returned error can't find the container with id a984c31b971b12da08a9d7e867a73f5cf3528455a2e51208739c17712450190d
Jul 15 21:02:12.359216 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.359198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wvkjb"
Jul 15 21:02:12.366108 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.366087    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45bcfe0_8661_44ec_8830_9e1dcaa5d8e3.slice/crio-a882523c59957f548401259c07d4882ce2ce737640de21666e0d7dad5d4257f3 WatchSource:0}: Error finding container a882523c59957f548401259c07d4882ce2ce737640de21666e0d7dad5d4257f3: Status 404 returned error can't find the container with id a882523c59957f548401259c07d4882ce2ce737640de21666e0d7dad5d4257f3
Jul 15 21:02:12.376535 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.376515    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:12.383724 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.383701    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ff2fdc_8161_49f8_add9_8393e61dd94f.slice/crio-082e7273cd625a3353dc614bac6f8a1ff54ce271f33e9f2dbb420ef4007d5e26 WatchSource:0}: Error finding container 082e7273cd625a3353dc614bac6f8a1ff54ce271f33e9f2dbb420ef4007d5e26: Status 404 returned error can't find the container with id 082e7273cd625a3353dc614bac6f8a1ff54ce271f33e9f2dbb420ef4007d5e26
Jul 15 21:02:12.395447 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.395428    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-599sr"
Jul 15 21:02:12.402690 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.402667    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855e91fe_3c9d_4f36_a073_09eecd10ed4c.slice/crio-80f7e7bde09cd6a2936378f5764f68cb6a4af989554982918d0a8f3be9659ba7 WatchSource:0}: Error finding container 80f7e7bde09cd6a2936378f5764f68cb6a4af989554982918d0a8f3be9659ba7: Status 404 returned error can't find the container with id 80f7e7bde09cd6a2936378f5764f68cb6a4af989554982918d0a8f3be9659ba7
Jul 15 21:02:12.407454 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.407438    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:02:12.420313 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.420291    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lc5h4"
Jul 15 21:02:12.424983 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.424897    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cbg28"
Jul 15 21:02:12.427179 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.427157    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb4db71_5cbe_49f6_9c8f_99fb32f52a4e.slice/crio-5a7cd027ed03346829733e2561958b917d394c685f5fd2fa0d080b538f6a330d WatchSource:0}: Error finding container 5a7cd027ed03346829733e2561958b917d394c685f5fd2fa0d080b538f6a330d: Status 404 returned error can't find the container with id 5a7cd027ed03346829733e2561958b917d394c685f5fd2fa0d080b538f6a330d
Jul 15 21:02:12.431836 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:12.431816    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod330f78ca_c79c_409c_9185_3f188ddb8086.slice/crio-fdb603d87938336b16edd7d7d59d4e53af2496c1a1d392c1a25eef0af32d9f40 WatchSource:0}: Error finding container fdb603d87938336b16edd7d7d59d4e53af2496c1a1d392c1a25eef0af32d9f40: Status 404 returned error can't find the container with id fdb603d87938336b16edd7d7d59d4e53af2496c1a1d392c1a25eef0af32d9f40
Jul 15 21:02:12.593287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.593199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:12.593446 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.593349    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:12.593446 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.593419    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret podName:4e184053-7574-434b-8873-db1b343c9589 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:13.593399471 +0000 UTC m=+2.950318049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret") pod "global-pull-secret-syncer-6ldsz" (UID: "4e184053-7574-434b-8873-db1b343c9589") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:12.694446 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.694412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:12.694989 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.694948    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:12.695091 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.695074    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs podName:e3220222-dfc1-4a57-a205-809e0fba860e nodeName:}" failed. No retries permitted until 2026-07-15 21:02:13.695053087 +0000 UTC m=+3.051971680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs") pod "network-metrics-daemon-xr98b" (UID: "e3220222-dfc1-4a57-a205-809e0fba860e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:12.795334 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:12.795291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:12.795576 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.795542    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:02:12.795576 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.795563    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:02:12.795576 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.795577    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nm9j9 for pod openshift-network-diagnostics/network-check-target-kmgwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:12.795743 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:12.795638    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9 podName:31bad667-66ee-41eb-9643-a72135c77989 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:13.795619352 +0000 UTC m=+3.152537941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nm9j9" (UniqueName: "kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9") pod "network-check-target-kmgwk" (UID: "31bad667-66ee-41eb-9643-a72135c77989") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:13.043419 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.043258    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 20:57:12 +0000 UTC" deadline="2028-04-05 05:17:41.05682105 +0000 UTC"
Jul 15 21:02:13.043419 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.043322    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15104h15m28.013503879s"
Jul 15 21:02:13.048863 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.048656    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:02:13.131457 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.131425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:13.131644 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:13.131571    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:13.159632 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.159593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cbg28" event={"ID":"330f78ca-c79c-409c-9185-3f188ddb8086","Type":"ContainerStarted","Data":"fdb603d87938336b16edd7d7d59d4e53af2496c1a1d392c1a25eef0af32d9f40"}
Jul 15 21:02:13.161948 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.161917    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lc5h4" event={"ID":"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e","Type":"ContainerStarted","Data":"5a7cd027ed03346829733e2561958b917d394c685f5fd2fa0d080b538f6a330d"}
Jul 15 21:02:13.165331 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.165305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-599sr" event={"ID":"855e91fe-3c9d-4f36-a073-09eecd10ed4c","Type":"ContainerStarted","Data":"80f7e7bde09cd6a2936378f5764f68cb6a4af989554982918d0a8f3be9659ba7"}
Jul 15 21:02:13.177921 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.177870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerStarted","Data":"082e7273cd625a3353dc614bac6f8a1ff54ce271f33e9f2dbb420ef4007d5e26"}
Jul 15 21:02:13.202434 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.202396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" event={"ID":"b1899eb2-d219-4554-910b-e0e59f2b22c2","Type":"ContainerStarted","Data":"a984c31b971b12da08a9d7e867a73f5cf3528455a2e51208739c17712450190d"}
Jul 15 21:02:13.205604 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.205574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qncr2" event={"ID":"96932186-bd45-4160-b3e7-70509123eff9","Type":"ContainerStarted","Data":"d627d60d3c805f37be7b4fd563d12c8cce94914b42d80babd1c34b61550e5347"}
Jul 15 21:02:13.209202 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.209170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg" event={"ID":"94b810f1-fbc9-418f-811f-186122b7027a","Type":"ContainerStarted","Data":"fbb62bc1c2b5488768b086c4019ae1d5f4450d2021722780e7292ab528dd8ca4"}
Jul 15 21:02:13.215706 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.215511    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wvkjb" event={"ID":"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3","Type":"ContainerStarted","Data":"a882523c59957f548401259c07d4882ce2ce737640de21666e0d7dad5d4257f3"}
Jul 15 21:02:13.217253 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.217222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nhsnr" event={"ID":"3517b0c1-318b-457b-b272-efc0e289dbbc","Type":"ContainerStarted","Data":"c3eadad065b3bf5df4a20514a61980754a9f8c98ee526646c420c65ca8aabb65"}
Jul 15 21:02:13.601780 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.601735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:13.602001 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:13.601889    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:13.602001 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:13.601952    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret podName:4e184053-7574-434b-8873-db1b343c9589 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:15.601934317 +0000 UTC m=+4.958852906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret") pod "global-pull-secret-syncer-6ldsz" (UID: "4e184053-7574-434b-8873-db1b343c9589") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:13.703210 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.703137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:13.703398 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:13.703298    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:13.703398 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:13.703364    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs podName:e3220222-dfc1-4a57-a205-809e0fba860e nodeName:}" failed. No retries permitted until 2026-07-15 21:02:15.703345634 +0000 UTC m=+5.060264226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs") pod "network-metrics-daemon-xr98b" (UID: "e3220222-dfc1-4a57-a205-809e0fba860e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:13.803882 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:13.803838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:13.804085 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:13.804047    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:02:13.804085 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:13.804067    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:02:13.804085 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:13.804079    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nm9j9 for pod openshift-network-diagnostics/network-check-target-kmgwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:13.804232 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:13.804139    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9 podName:31bad667-66ee-41eb-9643-a72135c77989 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:15.80412011 +0000 UTC m=+5.161038702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nm9j9" (UniqueName: "kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9") pod "network-check-target-kmgwk" (UID: "31bad667-66ee-41eb-9643-a72135c77989") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:14.043742 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:14.043604    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 20:57:12 +0000 UTC" deadline="2028-01-09 18:19:49.414266362 +0000 UTC"
Jul 15 21:02:14.043742 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:14.043648    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13029h17m35.370623092s"
Jul 15 21:02:14.128665 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:14.128231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:14.128665 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:14.128357    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:14.128665 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:14.128378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:14.128665 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:14.128486    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:14.738602 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:14.738538    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 21:02:15.129782 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:15.129741    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:15.130303 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:15.129894    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:15.619040 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:15.618938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:15.619278 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:15.619117    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:15.619278 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:15.619176    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret podName:4e184053-7574-434b-8873-db1b343c9589 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:19.61915794 +0000 UTC m=+8.976076516 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret") pod "global-pull-secret-syncer-6ldsz" (UID: "4e184053-7574-434b-8873-db1b343c9589") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:15.720252 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:15.720214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:15.720434 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:15.720363    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:15.720434 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:15.720426    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs podName:e3220222-dfc1-4a57-a205-809e0fba860e nodeName:}" failed. No retries permitted until 2026-07-15 21:02:19.72040553 +0000 UTC m=+9.077324112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs") pod "network-metrics-daemon-xr98b" (UID: "e3220222-dfc1-4a57-a205-809e0fba860e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:15.821078 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:15.821045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:15.821269 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:15.821195    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:02:15.821269 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:15.821216    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:02:15.821269 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:15.821229    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nm9j9 for pod openshift-network-diagnostics/network-check-target-kmgwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:15.821419 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:15.821287    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9 podName:31bad667-66ee-41eb-9643-a72135c77989 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:19.821268219 +0000 UTC m=+9.178186808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nm9j9" (UniqueName: "kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9") pod "network-check-target-kmgwk" (UID: "31bad667-66ee-41eb-9643-a72135c77989") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:16.128285 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:16.128255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:16.128473 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:16.128376    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:16.128473 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:16.128441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:16.128611 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:16.128511    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:17.129950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:17.129513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:17.129950 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:17.129698    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:18.128320 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:18.128284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:18.128509 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:18.128320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:18.128509 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:18.128429    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:18.128626 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:18.128580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:19.129085 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:19.129049    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:19.129525 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:19.129215    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:19.654471 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:19.654121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:19.654471 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:19.654326    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:19.654471 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:19.654398    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret podName:4e184053-7574-434b-8873-db1b343c9589 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:27.654377221 +0000 UTC m=+17.011295820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret") pod "global-pull-secret-syncer-6ldsz" (UID: "4e184053-7574-434b-8873-db1b343c9589") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:19.754896 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:19.754848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:19.755088 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:19.755049    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:19.755144 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:19.755133    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs podName:e3220222-dfc1-4a57-a205-809e0fba860e nodeName:}" failed. No retries permitted until 2026-07-15 21:02:27.755110327 +0000 UTC m=+17.112028904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs") pod "network-metrics-daemon-xr98b" (UID: "e3220222-dfc1-4a57-a205-809e0fba860e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:19.856245 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:19.856187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:19.856431 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:19.856413    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:02:19.856489 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:19.856436    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:02:19.856489 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:19.856448    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nm9j9 for pod openshift-network-diagnostics/network-check-target-kmgwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:19.856585 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:19.856508    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9 podName:31bad667-66ee-41eb-9643-a72135c77989 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:27.856489575 +0000 UTC m=+17.213408166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nm9j9" (UniqueName: "kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9") pod "network-check-target-kmgwk" (UID: "31bad667-66ee-41eb-9643-a72135c77989") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:20.128539 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:20.128442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:20.128708 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:20.128605    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:20.128999 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:20.128443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:20.129130 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:20.129050    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:21.130122 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:21.130077    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:21.130461 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:21.130209    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:22.128237 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:22.128198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:22.128425 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:22.128246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:22.128425 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:22.128333    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:22.128528 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:22.128487    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:23.128709 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:23.128671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:23.129185 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:23.128811    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:24.129022 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:24.128924    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:24.129472 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:24.128924    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:24.129472 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:24.129062    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:24.129472 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:24.129141    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:25.128864 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:25.128819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:25.129076 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:25.128982    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:26.128186 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:26.128150    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:26.128368 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:26.128282    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:26.128368 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:26.128328    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:26.128534 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:26.128491    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:27.128567 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:27.128524    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:27.129020 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:27.128687    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:27.719470 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:27.719434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:27.719675 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:27.719607    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:27.719728 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:27.719679    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret podName:4e184053-7574-434b-8873-db1b343c9589 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:43.719657698 +0000 UTC m=+33.076576279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret") pod "global-pull-secret-syncer-6ldsz" (UID: "4e184053-7574-434b-8873-db1b343c9589") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:27.820446 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:27.820409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:27.820607 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:27.820533    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:27.820607 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:27.820595    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs podName:e3220222-dfc1-4a57-a205-809e0fba860e nodeName:}" failed. No retries permitted until 2026-07-15 21:02:43.820578861 +0000 UTC m=+33.177497458 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs") pod "network-metrics-daemon-xr98b" (UID: "e3220222-dfc1-4a57-a205-809e0fba860e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:27.920980 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:27.920882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:27.921134 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:27.921092    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:02:27.921134 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:27.921118    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:02:27.921134 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:27.921133    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nm9j9 for pod openshift-network-diagnostics/network-check-target-kmgwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:27.921266 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:27.921192    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9 podName:31bad667-66ee-41eb-9643-a72135c77989 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:43.921177911 +0000 UTC m=+33.278096485 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nm9j9" (UniqueName: "kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9") pod "network-check-target-kmgwk" (UID: "31bad667-66ee-41eb-9643-a72135c77989") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:28.128292 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:28.128252    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:28.128468 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:28.128375    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:28.128468 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:28.128436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:28.128568 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:28.128549    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:29.128657 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:29.128616    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:29.129166 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:29.128762    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:30.129025 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:30.128813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:30.129459 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:30.128812    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:30.129459 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:30.129129    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:30.129459 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:30.129183    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:31.129841 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:31.129800    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:31.130295 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:31.129936    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:32.128914 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:32.128839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:32.129077 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:32.128839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:32.129077 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:32.128982    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:32.129077 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:32.129038    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:33.128599 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.128301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:33.129411 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:33.128710    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:33.261434 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.261393    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wvkjb" event={"ID":"e45bcfe0-8661-44ec-8830-9e1dcaa5d8e3","Type":"ContainerStarted","Data":"a1b237dcac5c41472b6e8065f6f50ad995dabb6389a5352e5f1270b05ea6db52"}
Jul 15 21:02:33.263573 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.263543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nhsnr" event={"ID":"3517b0c1-318b-457b-b272-efc0e289dbbc","Type":"ContainerStarted","Data":"71aa1f679112644a1a4c2c429766ab33ff41257e5a4a890a2818b54167619a5e"}
Jul 15 21:02:33.265264 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.265231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal" event={"ID":"46ea3c93a970f7700a2a3849415d7a22","Type":"ContainerStarted","Data":"8138c8703a3f31758f1e88dc493e63ec1049f78617f7a33a00b4d071e4e24377"}
Jul 15 21:02:33.266843 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.266820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lc5h4" event={"ID":"0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e","Type":"ContainerStarted","Data":"895abd61dc4572ed367324e4d1d5abc95431adbd9581b7a70b0d8af824724a82"}
Jul 15 21:02:33.268843 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.268713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-599sr" event={"ID":"855e91fe-3c9d-4f36-a073-09eecd10ed4c","Type":"ContainerStarted","Data":"7ed1db5c77d1123539df0b2bcca59358af1225d79e2a462d7de254145eda1825"}
Jul 15 21:02:33.275944 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.275924    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:02:33.276355 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.276312    2580 generic.go:356] "Generic (PLEG): container finished" podID="a5ff2fdc-8161-49f8-add9-8393e61dd94f" containerID="2378b43caff216b0c812ac087bec0130c27a0c6dd10a5833d9afac4ff33cd3a1" exitCode=1
Jul 15 21:02:33.276432 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.276377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerStarted","Data":"2f65fd51c49d378513ec9dfcf74dfd017a720da8539e82a963736f639c6ed02c"}
Jul 15 21:02:33.276492 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.276457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerStarted","Data":"795551d6a8cd2218884481eb6d34c6b027c5d3146f321de3ed7b73fb017136b5"}
Jul 15 21:02:33.276563 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.276507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerStarted","Data":"fb9653ef49d238bbd8ae432f8941d6a75409d64461d377349d133f6c9c5e95c0"}
Jul 15 21:02:33.276563 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.276547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerStarted","Data":"76cc71330f8ce401d52e2c788b67ee3f77a3b4920f0bf88b1bed4cc9e774e2b2"}
Jul 15 21:02:33.276654 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.276576    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerDied","Data":"2378b43caff216b0c812ac087bec0130c27a0c6dd10a5833d9afac4ff33cd3a1"}
Jul 15 21:02:33.276654 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.276614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerStarted","Data":"2ca520af90e33513a13790917d56b9a0ddae56925ad5bc81a757e8dcd06dee63"}
Jul 15 21:02:33.277994 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.277950    2580 generic.go:356] "Generic (PLEG): container finished" podID="b1899eb2-d219-4554-910b-e0e59f2b22c2" containerID="6305440cf269ec8e519aaf1172a5c31ff490f6f22582f511eec00e7754f67cdb" exitCode=0
Jul 15 21:02:33.277994 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.277988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" event={"ID":"b1899eb2-d219-4554-910b-e0e59f2b22c2","Type":"ContainerDied","Data":"6305440cf269ec8e519aaf1172a5c31ff490f6f22582f511eec00e7754f67cdb"}
Jul 15 21:02:33.279969 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.279931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qncr2" event={"ID":"96932186-bd45-4160-b3e7-70509123eff9","Type":"ContainerStarted","Data":"1c39cc0c8633405d6d06d2ee1e4c0c2d759df0c4066b6a09e908abf67eee852d"}
Jul 15 21:02:33.281461 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.281425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg" event={"ID":"94b810f1-fbc9-418f-811f-186122b7027a","Type":"ContainerStarted","Data":"4c8b82a7f78401ea2f53877cc458c776339a64c2979badbdeb118a8bfcd96c16"}
Jul 15 21:02:33.283096 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.283046    2580 generic.go:356] "Generic (PLEG): container finished" podID="4e866a2a89b76039c681f32045b5640c" containerID="2e050546d26f9daea24f1e1d1da74eadf0c24af05c6a09765a202000ecc98b08" exitCode=0
Jul 15 21:02:33.283096 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.283082    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal" event={"ID":"4e866a2a89b76039c681f32045b5640c","Type":"ContainerDied","Data":"2e050546d26f9daea24f1e1d1da74eadf0c24af05c6a09765a202000ecc98b08"}
Jul 15 21:02:33.283432 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.283396    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wvkjb" podStartSLOduration=2.20543645 podStartE2EDuration="22.283381652s" totalImagesPullingTime="20.077945202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:33.282815588 +0000 UTC m=+22.639734186" watchObservedRunningTime="2026-07-15 21:02:33.283381652 +0000 UTC m=+22.640300250"
Jul 15 21:02:33.299300 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.299262    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qncr2" podStartSLOduration=2.2664668900000002 podStartE2EDuration="22.299248356s" totalImagesPullingTime="20.032781466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:33.299157258 +0000 UTC m=+22.656075855" watchObservedRunningTime="2026-07-15 21:02:33.299248356 +0000 UTC m=+22.656166984"
Jul 15 21:02:33.337017 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.336938    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-599sr" podStartSLOduration=2.363255056 podStartE2EDuration="22.336922428s" totalImagesPullingTime="19.973667372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:33.336510279 +0000 UTC m=+22.693428879" watchObservedRunningTime="2026-07-15 21:02:33.336922428 +0000 UTC m=+22.693841025"
Jul 15 21:02:33.355367 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.355325    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lc5h4" podStartSLOduration=2.435564836 podStartE2EDuration="22.355310455s" totalImagesPullingTime="19.919745619s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:33.354578016 +0000 UTC m=+22.711496617" watchObservedRunningTime="2026-07-15 21:02:33.355310455 +0000 UTC m=+22.712229051"
Jul 15 21:02:33.383082 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.383033    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-29.ec2.internal" podStartSLOduration=21.383018681 podStartE2EDuration="21.383018681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:33.3825259 +0000 UTC m=+22.739444479" watchObservedRunningTime="2026-07-15 21:02:33.383018681 +0000 UTC m=+22.739937277"
Jul 15 21:02:33.416381 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.416319    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nhsnr" podStartSLOduration=2.882113956 podStartE2EDuration="22.416297906s" totalImagesPullingTime="19.53418395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:33.415192867 +0000 UTC m=+22.772111465" watchObservedRunningTime="2026-07-15 21:02:33.416297906 +0000 UTC m=+22.773216503"
Jul 15 21:02:33.875520 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:33.875490    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 21:02:34.052306 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.052168    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T21:02:33.875513971Z","UUID":"25c08f73-6f69-4120-96b7-670dab345056","Handler":null,"Name":"","Endpoint":""}
Jul 15 21:02:34.053769 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.053746    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 21:02:34.053899 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.053787    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 21:02:34.128179 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.128142    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:34.128356 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.128142    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:34.128356 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:34.128288    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:34.128470 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:34.128359    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:34.290220 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.290185    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cbg28" event={"ID":"330f78ca-c79c-409c-9185-3f188ddb8086","Type":"ContainerStarted","Data":"5762456461dd409c76d4e3bcc631e1542bc58c98d2ffe1346c98d5a6dd3cdad0"}
Jul 15 21:02:34.292460 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.292429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg" event={"ID":"94b810f1-fbc9-418f-811f-186122b7027a","Type":"ContainerStarted","Data":"4eda9c345964dbfe19d135395acbae4fdad18be8af1a2dd08e05774ff217df23"}
Jul 15 21:02:34.294641 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.294610    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal" event={"ID":"4e866a2a89b76039c681f32045b5640c","Type":"ContainerStarted","Data":"f1aa7039f3182b538878f7b8f4bd480b53cf503986150d4cb3c54d1e0a945160"}
Jul 15 21:02:34.308562 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.308456    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cbg28" podStartSLOduration=3.370131372 podStartE2EDuration="23.308439685s" totalImagesPullingTime="19.938308313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:34.308097203 +0000 UTC m=+23.665015800" watchObservedRunningTime="2026-07-15 21:02:34.308439685 +0000 UTC m=+23.665358283"
Jul 15 21:02:34.324537 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:34.324483    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-29.ec2.internal" podStartSLOduration=23.324461772 podStartE2EDuration="23.324461772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:34.323927022 +0000 UTC m=+23.680845620" watchObservedRunningTime="2026-07-15 21:02:34.324461772 +0000 UTC m=+23.681380370"
Jul 15 21:02:35.129041 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:35.129006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:35.129232 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:35.129151    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:35.300271 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:35.300234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg" event={"ID":"94b810f1-fbc9-418f-811f-186122b7027a","Type":"ContainerStarted","Data":"00a1b0f9f12d666b7928515882b55e34ca56b46c3c7ffbef345ec8ecb344f962"}
Jul 15 21:02:35.322027 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:35.321949    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlmwg" podStartSLOduration=2.7382725199999998 podStartE2EDuration="24.321929081s" totalImagesPullingTime="21.583656561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:35.321792283 +0000 UTC m=+24.678710881" watchObservedRunningTime="2026-07-15 21:02:35.321929081 +0000 UTC m=+24.678847679"
Jul 15 21:02:35.663369 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:35.663286    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:35.663872 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:35.663850    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:36.128519 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:36.128284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:36.128691 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:36.128288    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:36.128691 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:36.128563    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:36.128691 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:36.128662    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:36.306405 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:36.306379    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:02:36.306922 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:36.306772    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerStarted","Data":"cbc0412376483437acd67459bc256b4899b7f2ac1edb321d42c380e5d98ac723"}
Jul 15 21:02:36.307240 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:36.307194    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:36.308163 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:36.308143    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nhsnr"
Jul 15 21:02:37.128504 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:37.128469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:37.128854 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:37.128613    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:38.129124 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.128924    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:38.129800 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.128996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:38.129800 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:38.129209    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:38.129800 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:38.129264    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:38.315247 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.315177    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:02:38.315549 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.315528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerStarted","Data":"d26332abef7f521e6baf3ac3fc64ddefc58a040d586d543ad3d3769a2218fde7"}
Jul 15 21:02:38.315716 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.315686    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:38.315847 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.315786    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:38.315847 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.315834    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:38.316029 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.316013    2580 scope.go:117] "RemoveContainer" containerID="2378b43caff216b0c812ac087bec0130c27a0c6dd10a5833d9afac4ff33cd3a1"
Jul 15 21:02:38.317748 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.317649    2580 generic.go:356] "Generic (PLEG): container finished" podID="b1899eb2-d219-4554-910b-e0e59f2b22c2" containerID="adf6197d0694a98c91273637c764489a582d763d570b0a32b009543e29b65452" exitCode=0
Jul 15 21:02:38.317748 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.317695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" event={"ID":"b1899eb2-d219-4554-910b-e0e59f2b22c2","Type":"ContainerDied","Data":"adf6197d0694a98c91273637c764489a582d763d570b0a32b009543e29b65452"}
Jul 15 21:02:38.334104 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.334076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:38.334766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:38.334745    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:02:39.128137 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.128107    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:39.128367 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:39.128225    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:39.324847 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.324807    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:02:39.325298 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.325275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" event={"ID":"a5ff2fdc-8161-49f8-add9-8393e61dd94f","Type":"ContainerStarted","Data":"7ce741fc0e4f3b782ad2ce64089c512a61fa4a85ad8726a13b148617b313e6c1"}
Jul 15 21:02:39.357900 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.357715    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk" podStartSLOduration=8.359031565 podStartE2EDuration="28.357700185s" totalImagesPullingTime="19.99866862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:39.35619587 +0000 UTC m=+28.713114467" watchObservedRunningTime="2026-07-15 21:02:39.357700185 +0000 UTC m=+28.714618808"
Jul 15 21:02:39.491024 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.490929    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6ldsz"]
Jul 15 21:02:39.491174 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.491052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:39.491174 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:39.491131    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:39.498077 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.498053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xr98b"]
Jul 15 21:02:39.498191 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.498137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:39.498236 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:39.498217    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:39.512507 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.512481    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kmgwk"]
Jul 15 21:02:39.512622 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:39.512583    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:39.512688 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:39.512672    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:40.329969 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:40.329935    2580 generic.go:356] "Generic (PLEG): container finished" podID="b1899eb2-d219-4554-910b-e0e59f2b22c2" containerID="2bc0795e9494a3b08b8efcb8b7c39cd563e6040040a4b3ffd9db06d7c72633c8" exitCode=0
Jul 15 21:02:40.330400 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:40.330016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" event={"ID":"b1899eb2-d219-4554-910b-e0e59f2b22c2","Type":"ContainerDied","Data":"2bc0795e9494a3b08b8efcb8b7c39cd563e6040040a4b3ffd9db06d7c72633c8"}
Jul 15 21:02:41.129803 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:41.129767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:41.130026 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:41.129849    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:41.130026 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:41.129856    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:41.130026 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:41.129976    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:41.130207 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:41.130024    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:41.130207 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:41.130154    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:41.334673 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:41.334645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" event={"ID":"b1899eb2-d219-4554-910b-e0e59f2b22c2","Type":"ContainerStarted","Data":"154bc0f4cb44104a9eb92e3e7b380b240532c3567b6e11b3ed36783ec88c3f27"}
Jul 15 21:02:42.339409 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:42.339373    2580 generic.go:356] "Generic (PLEG): container finished" podID="b1899eb2-d219-4554-910b-e0e59f2b22c2" containerID="154bc0f4cb44104a9eb92e3e7b380b240532c3567b6e11b3ed36783ec88c3f27" exitCode=0
Jul 15 21:02:42.339981 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:42.339420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" event={"ID":"b1899eb2-d219-4554-910b-e0e59f2b22c2","Type":"ContainerDied","Data":"154bc0f4cb44104a9eb92e3e7b380b240532c3567b6e11b3ed36783ec88c3f27"}
Jul 15 21:02:43.128904 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:43.128873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:43.129107 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:43.128873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:43.129107 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.128999    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ldsz" podUID="4e184053-7574-434b-8873-db1b343c9589"
Jul 15 21:02:43.129107 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:43.128873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:43.129107 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.129078    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xr98b" podUID="e3220222-dfc1-4a57-a205-809e0fba860e"
Jul 15 21:02:43.129278 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.129141    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmgwk" podUID="31bad667-66ee-41eb-9643-a72135c77989"
Jul 15 21:02:43.737524 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:43.737468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:43.738138 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.737657    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:43.738138 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.737742    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret podName:4e184053-7574-434b-8873-db1b343c9589 nodeName:}" failed. No retries permitted until 2026-07-15 21:03:15.737720127 +0000 UTC m=+65.094638702 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret") pod "global-pull-secret-syncer-6ldsz" (UID: "4e184053-7574-434b-8873-db1b343c9589") : object "kube-system"/"original-pull-secret" not registered
Jul 15 21:02:43.838291 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:43.838253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:43.838466 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.838379    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:43.838466 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.838452    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs podName:e3220222-dfc1-4a57-a205-809e0fba860e nodeName:}" failed. No retries permitted until 2026-07-15 21:03:15.838434621 +0000 UTC m=+65.195353209 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs") pod "network-metrics-daemon-xr98b" (UID: "e3220222-dfc1-4a57-a205-809e0fba860e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 21:02:43.939472 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:43.939433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:43.939664 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.939584    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 21:02:43.939664 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.939603    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 21:02:43.939664 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.939611    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nm9j9 for pod openshift-network-diagnostics/network-check-target-kmgwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:43.939664 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:43.939663    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9 podName:31bad667-66ee-41eb-9643-a72135c77989 nodeName:}" failed. No retries permitted until 2026-07-15 21:03:15.939650154 +0000 UTC m=+65.296568743 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nm9j9" (UniqueName: "kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9") pod "network-check-target-kmgwk" (UID: "31bad667-66ee-41eb-9643-a72135c77989") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 21:02:45.024899 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.024871    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-29.ec2.internal" event="NodeReady"
Jul 15 21:02:45.025357 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.025052    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 21:02:45.079319 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.079282    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-79q2z"]
Jul 15 21:02:45.081913 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.081883    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jv7fg"]
Jul 15 21:02:45.082061 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.082034    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.083525 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.083502    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jv7fg"
Jul 15 21:02:45.085869 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.085849    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 21:02:45.085992 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.085872    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 21:02:45.086098 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.086069    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 21:02:45.086098 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.086083    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8p5p2\""
Jul 15 21:02:45.086208 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.086128    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 21:02:45.086572 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.086555    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p4nfw\""
Jul 15 21:02:45.086665 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.086634    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 21:02:45.096299 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.096228    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-79q2z"]
Jul 15 21:02:45.097101 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.097024    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jv7fg"]
Jul 15 21:02:45.128858 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.128825    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:02:45.129043 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.128839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:02:45.129119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.129056    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:02:45.131978 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.131937    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 21:02:45.131978 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.131950    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rwswm\""
Jul 15 21:02:45.132156 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.131978    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 21:02:45.132156 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.132005    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfzc2\""
Jul 15 21:02:45.132156 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.131942    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 21:02:45.132156 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.132085    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 21:02:45.250053 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.249948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5bef743-7548-4f72-8caa-6798e573e30b-tmp-dir\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.250227 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.250055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k99n9\" (UniqueName: \"kubernetes.io/projected/8da41a2e-7aef-4cc5-87a1-05a9570f8642-kube-api-access-k99n9\") pod \"ingress-canary-jv7fg\" (UID: \"8da41a2e-7aef-4cc5-87a1-05a9570f8642\") " pod="openshift-ingress-canary/ingress-canary-jv7fg"
Jul 15 21:02:45.250227 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.250149    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bef743-7548-4f72-8caa-6798e573e30b-config-volume\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.250330 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.250313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8da41a2e-7aef-4cc5-87a1-05a9570f8642-cert\") pod \"ingress-canary-jv7fg\" (UID: \"8da41a2e-7aef-4cc5-87a1-05a9570f8642\") " pod="openshift-ingress-canary/ingress-canary-jv7fg"
Jul 15 21:02:45.250384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.250355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.250438 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.250411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzckw\" (UniqueName: \"kubernetes.io/projected/c5bef743-7548-4f72-8caa-6798e573e30b-kube-api-access-tzckw\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.351758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.351724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzckw\" (UniqueName: \"kubernetes.io/projected/c5bef743-7548-4f72-8caa-6798e573e30b-kube-api-access-tzckw\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.351950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.351904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5bef743-7548-4f72-8caa-6798e573e30b-tmp-dir\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.351950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.351947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k99n9\" (UniqueName: \"kubernetes.io/projected/8da41a2e-7aef-4cc5-87a1-05a9570f8642-kube-api-access-k99n9\") pod \"ingress-canary-jv7fg\" (UID: \"8da41a2e-7aef-4cc5-87a1-05a9570f8642\") " pod="openshift-ingress-canary/ingress-canary-jv7fg"
Jul 15 21:02:45.352088 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.352009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bef743-7548-4f72-8caa-6798e573e30b-config-volume\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.352088 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.352063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8da41a2e-7aef-4cc5-87a1-05a9570f8642-cert\") pod \"ingress-canary-jv7fg\" (UID: \"8da41a2e-7aef-4cc5-87a1-05a9570f8642\") " pod="openshift-ingress-canary/ingress-canary-jv7fg"
Jul 15 21:02:45.352188 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.352093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.352239 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:45.352206    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:02:45.352288 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:45.352272    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls podName:c5bef743-7548-4f72-8caa-6798e573e30b nodeName:}" failed. No retries permitted until 2026-07-15 21:02:45.852252645 +0000 UTC m=+35.209171223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls") pod "dns-default-79q2z" (UID: "c5bef743-7548-4f72-8caa-6798e573e30b") : secret "dns-default-metrics-tls" not found
Jul 15 21:02:45.352347 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.352307    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5bef743-7548-4f72-8caa-6798e573e30b-tmp-dir\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.352650 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.352622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5bef743-7548-4f72-8caa-6798e573e30b-config-volume\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.356646 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.356493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8da41a2e-7aef-4cc5-87a1-05a9570f8642-cert\") pod \"ingress-canary-jv7fg\" (UID: \"8da41a2e-7aef-4cc5-87a1-05a9570f8642\") " pod="openshift-ingress-canary/ingress-canary-jv7fg"
Jul 15 21:02:45.363228 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.363181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzckw\" (UniqueName: \"kubernetes.io/projected/c5bef743-7548-4f72-8caa-6798e573e30b-kube-api-access-tzckw\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.363464 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.363442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k99n9\" (UniqueName: \"kubernetes.io/projected/8da41a2e-7aef-4cc5-87a1-05a9570f8642-kube-api-access-k99n9\") pod \"ingress-canary-jv7fg\" (UID: \"8da41a2e-7aef-4cc5-87a1-05a9570f8642\") " pod="openshift-ingress-canary/ingress-canary-jv7fg"
Jul 15 21:02:45.402386 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.402352    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jv7fg"
Jul 15 21:02:45.854966 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:45.854903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:45.855167 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:45.855081    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:02:45.855167 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:45.855160    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls podName:c5bef743-7548-4f72-8caa-6798e573e30b nodeName:}" failed. No retries permitted until 2026-07-15 21:02:46.855139502 +0000 UTC m=+36.212058091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls") pod "dns-default-79q2z" (UID: "c5bef743-7548-4f72-8caa-6798e573e30b") : secret "dns-default-metrics-tls" not found
Jul 15 21:02:46.050014 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.048097    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"]
Jul 15 21:02:46.051606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.051577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:46.053849 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.053823    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 15 21:02:46.053987 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.053832    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 15 21:02:46.054337 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.054321    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gblcx\""
Jul 15 21:02:46.054397 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.054376    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 15 21:02:46.058781 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.058752    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"]
Jul 15 21:02:46.153888 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.153809    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-mjds2"]
Jul 15 21:02:46.155978 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.155940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.156516 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.156491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:46.156621 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.156567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk6qj\" (UniqueName: \"kubernetes.io/projected/dfbccdef-809e-44d4-8970-cfbd4be88d33-kube-api-access-tk6qj\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:46.158717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.158694    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 15 21:02:46.158840 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.158788    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rq5b4\""
Jul 15 21:02:46.159242 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.159221    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 15 21:02:46.159337 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.159317    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 15 21:02:46.160012 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.159990    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 15 21:02:46.176364 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.176342    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 15 21:02:46.212109 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.212077    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-mjds2"]
Jul 15 21:02:46.257480 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.257445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s4z8\" (UniqueName: \"kubernetes.io/projected/72014806-cd64-45ba-be82-a71c7d7076fb-kube-api-access-6s4z8\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.257682 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.257505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:46.257682 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.257608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk6qj\" (UniqueName: \"kubernetes.io/projected/dfbccdef-809e-44d4-8970-cfbd4be88d33-kube-api-access-tk6qj\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:46.257682 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.257641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/72014806-cd64-45ba-be82-a71c7d7076fb-config\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.257682 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:46.257614    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 21:02:46.258004 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:46.257755    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls podName:dfbccdef-809e-44d4-8970-cfbd4be88d33 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:46.757734756 +0000 UTC m=+36.114653346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls") pod "cluster-samples-operator-65db858894-nhcz9" (UID: "dfbccdef-809e-44d4-8970-cfbd4be88d33") : secret "samples-operator-tls" not found
Jul 15 21:02:46.258004 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.257804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/72014806-cd64-45ba-be82-a71c7d7076fb-serving-cert\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.258004 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.257850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72014806-cd64-45ba-be82-a71c7d7076fb-trusted-ca\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.271025 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.270993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk6qj\" (UniqueName: \"kubernetes.io/projected/dfbccdef-809e-44d4-8970-cfbd4be88d33-kube-api-access-tk6qj\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:46.299899 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.299869    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"]
Jul 15 21:02:46.301494 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.301477    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.303579 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.303558    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d8f8c6575-98f4q"]
Jul 15 21:02:46.305056 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.305036    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 15 21:02:46.305340 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.305308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.305455 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.305431    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 15 21:02:46.305455 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.305449    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 15 21:02:46.308631 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.308576    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-lllmn\""
Jul 15 21:02:46.308946 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.308924    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 15 21:02:46.309943 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.309924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 21:02:46.310207 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.310140    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 21:02:46.310207 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.310189    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d59m9\""
Jul 15 21:02:46.310713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.310697    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 21:02:46.320047 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.320027    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 21:02:46.324080 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.324060    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"]
Jul 15 21:02:46.337035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.337001    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8f8c6575-98f4q"]
Jul 15 21:02:46.359133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.359089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/72014806-cd64-45ba-be82-a71c7d7076fb-config\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.359904 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.359880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/72014806-cd64-45ba-be82-a71c7d7076fb-config\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.360033 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.359909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/72014806-cd64-45ba-be82-a71c7d7076fb-serving-cert\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.360033 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.359942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72014806-cd64-45ba-be82-a71c7d7076fb-trusted-ca\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.360033 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.360017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s4z8\" (UniqueName: \"kubernetes.io/projected/72014806-cd64-45ba-be82-a71c7d7076fb-kube-api-access-6s4z8\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.361827 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.361803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72014806-cd64-45ba-be82-a71c7d7076fb-trusted-ca\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.371370 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.371339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/72014806-cd64-45ba-be82-a71c7d7076fb-serving-cert\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.390293 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.390266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s4z8\" (UniqueName: \"kubernetes.io/projected/72014806-cd64-45ba-be82-a71c7d7076fb-kube-api-access-6s4z8\") pod \"console-operator-7866497494-mjds2\" (UID: \"72014806-cd64-45ba-be82-a71c7d7076fb\") " pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.451642 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.451609    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jv7fg"]
Jul 15 21:02:46.455647 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:46.455611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8da41a2e_7aef_4cc5_87a1_05a9570f8642.slice/crio-00cd386e1978cbf817bcbad95992c7bbfd73ecd3587efcec3e7ad816937b5250 WatchSource:0}: Error finding container 00cd386e1978cbf817bcbad95992c7bbfd73ecd3587efcec3e7ad816937b5250: Status 404 returned error can't find the container with id 00cd386e1978cbf817bcbad95992c7bbfd73ecd3587efcec3e7ad816937b5250
Jul 15 21:02:46.460755 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.460730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.460841 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.460767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-bound-sa-token\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.460841 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.460802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkqfz\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-kube-api-access-qkqfz\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.460910 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.460889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8vw4\" (UniqueName: \"kubernetes.io/projected/1d90ed07-ed6a-458d-956a-1da3260948d2-kube-api-access-b8vw4\") pod \"kube-storage-version-migrator-operator-799cb7f958-dx82l\" (UID: \"1d90ed07-ed6a-458d-956a-1da3260948d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.460942 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.460924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-image-registry-private-configuration\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.461001 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.460969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d90ed07-ed6a-458d-956a-1da3260948d2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-dx82l\" (UID: \"1d90ed07-ed6a-458d-956a-1da3260948d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.461049 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.461036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-certificates\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.461084 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.461059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-trusted-ca\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.461115 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.461094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72565261-8a62-4c4e-9290-ce35e8e1abad-ca-trust-extracted\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.461115 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.461108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-installation-pull-secrets\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.461176 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.461125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d90ed07-ed6a-458d-956a-1da3260948d2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-dx82l\" (UID: \"1d90ed07-ed6a-458d-956a-1da3260948d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.466848 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.466826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:46.561623 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.561437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-certificates\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.561799 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.561640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-trusted-ca\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.561799 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.561694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72565261-8a62-4c4e-9290-ce35e8e1abad-ca-trust-extracted\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.561913 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.561815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-installation-pull-secrets\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.561913 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.561862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d90ed07-ed6a-458d-956a-1da3260948d2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-dx82l\" (UID: \"1d90ed07-ed6a-458d-956a-1da3260948d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.562076 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.562137 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-bound-sa-token\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.562188 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-certificates\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.562188 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkqfz\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-kube-api-access-qkqfz\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.562279 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8vw4\" (UniqueName: \"kubernetes.io/projected/1d90ed07-ed6a-458d-956a-1da3260948d2-kube-api-access-b8vw4\") pod \"kube-storage-version-migrator-operator-799cb7f958-dx82l\" (UID: \"1d90ed07-ed6a-458d-956a-1da3260948d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.562279 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:46.562214    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 21:02:46.562279 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:46.562233    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8f8c6575-98f4q: secret "image-registry-tls" not found
Jul 15 21:02:46.562279 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562232    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-image-registry-private-configuration\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.562279 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d90ed07-ed6a-458d-956a-1da3260948d2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-dx82l\" (UID: \"1d90ed07-ed6a-458d-956a-1da3260948d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.562497 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:46.562304    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls podName:72565261-8a62-4c4e-9290-ce35e8e1abad nodeName:}" failed. No retries permitted until 2026-07-15 21:02:47.062284153 +0000 UTC m=+36.419202749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls") pod "image-registry-d8f8c6575-98f4q" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad") : secret "image-registry-tls" not found
Jul 15 21:02:46.562497 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72565261-8a62-4c4e-9290-ce35e8e1abad-ca-trust-extracted\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.562800 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d90ed07-ed6a-458d-956a-1da3260948d2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-dx82l\" (UID: \"1d90ed07-ed6a-458d-956a-1da3260948d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.562895 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.562819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-trusted-ca\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.565171 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.565145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1d90ed07-ed6a-458d-956a-1da3260948d2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-dx82l\" (UID: \"1d90ed07-ed6a-458d-956a-1da3260948d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.565261 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.565232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-installation-pull-secrets\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.565369 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.565345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-image-registry-private-configuration\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.571103 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.571076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8vw4\" (UniqueName: \"kubernetes.io/projected/1d90ed07-ed6a-458d-956a-1da3260948d2-kube-api-access-b8vw4\") pod \"kube-storage-version-migrator-operator-799cb7f958-dx82l\" (UID: \"1d90ed07-ed6a-458d-956a-1da3260948d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.571274 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.571256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkqfz\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-kube-api-access-qkqfz\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.571489 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.571463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-bound-sa-token\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:46.592157 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.592108    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-mjds2"]
Jul 15 21:02:46.595280 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:46.595249    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72014806_cd64_45ba_be82_a71c7d7076fb.slice/crio-d20b63227ac0a28376862ff539c29fd6675c7f7204d01ffddb3cddcdfda77a93 WatchSource:0}: Error finding container d20b63227ac0a28376862ff539c29fd6675c7f7204d01ffddb3cddcdfda77a93: Status 404 returned error can't find the container with id d20b63227ac0a28376862ff539c29fd6675c7f7204d01ffddb3cddcdfda77a93
Jul 15 21:02:46.614508 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.614479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"
Jul 15 21:02:46.747891 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.747863    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l"]
Jul 15 21:02:46.751095 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:46.751062    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d90ed07_ed6a_458d_956a_1da3260948d2.slice/crio-c4750c5d2612c9d7e6393b802ba216a9344f2326e32fead0cb49d6ba8f64c997 WatchSource:0}: Error finding container c4750c5d2612c9d7e6393b802ba216a9344f2326e32fead0cb49d6ba8f64c997: Status 404 returned error can't find the container with id c4750c5d2612c9d7e6393b802ba216a9344f2326e32fead0cb49d6ba8f64c997
Jul 15 21:02:46.764138 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.764105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:46.764262 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:46.764245    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 21:02:46.764317 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:46.764303    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls podName:dfbccdef-809e-44d4-8970-cfbd4be88d33 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:47.76428457 +0000 UTC m=+37.121203160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls") pod "cluster-samples-operator-65db858894-nhcz9" (UID: "dfbccdef-809e-44d4-8970-cfbd4be88d33") : secret "samples-operator-tls" not found
Jul 15 21:02:46.864949 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:46.864912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:46.865137 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:46.865065    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:02:46.865137 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:46.865127    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls podName:c5bef743-7548-4f72-8caa-6798e573e30b nodeName:}" failed. No retries permitted until 2026-07-15 21:02:48.865109948 +0000 UTC m=+38.222028540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls") pod "dns-default-79q2z" (UID: "c5bef743-7548-4f72-8caa-6798e573e30b") : secret "dns-default-metrics-tls" not found
Jul 15 21:02:47.067022 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:47.066937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:47.067484 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:47.067083    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 21:02:47.067484 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:47.067101    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8f8c6575-98f4q: secret "image-registry-tls" not found
Jul 15 21:02:47.067484 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:47.067159    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls podName:72565261-8a62-4c4e-9290-ce35e8e1abad nodeName:}" failed. No retries permitted until 2026-07-15 21:02:48.067140227 +0000 UTC m=+37.424058820 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls") pod "image-registry-d8f8c6575-98f4q" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad") : secret "image-registry-tls" not found
Jul 15 21:02:47.354141 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:47.354052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l" event={"ID":"1d90ed07-ed6a-458d-956a-1da3260948d2","Type":"ContainerStarted","Data":"c4750c5d2612c9d7e6393b802ba216a9344f2326e32fead0cb49d6ba8f64c997"}
Jul 15 21:02:47.355322 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:47.355287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-mjds2" event={"ID":"72014806-cd64-45ba-be82-a71c7d7076fb","Type":"ContainerStarted","Data":"d20b63227ac0a28376862ff539c29fd6675c7f7204d01ffddb3cddcdfda77a93"}
Jul 15 21:02:47.356518 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:47.356487    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jv7fg" event={"ID":"8da41a2e-7aef-4cc5-87a1-05a9570f8642","Type":"ContainerStarted","Data":"00cd386e1978cbf817bcbad95992c7bbfd73ecd3587efcec3e7ad816937b5250"}
Jul 15 21:02:47.359582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:47.359553    2580 generic.go:356] "Generic (PLEG): container finished" podID="b1899eb2-d219-4554-910b-e0e59f2b22c2" containerID="df656fc448771c7156c2e8a81b3073a616beb16c38f2e2fb3a358b8940c325a5" exitCode=0
Jul 15 21:02:47.359695 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:47.359606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" event={"ID":"b1899eb2-d219-4554-910b-e0e59f2b22c2","Type":"ContainerDied","Data":"df656fc448771c7156c2e8a81b3073a616beb16c38f2e2fb3a358b8940c325a5"}
Jul 15 21:02:47.774123 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:47.774036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:47.774274 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:47.774219    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 21:02:47.774326 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:47.774290    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls podName:dfbccdef-809e-44d4-8970-cfbd4be88d33 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:49.774275003 +0000 UTC m=+39.131193578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls") pod "cluster-samples-operator-65db858894-nhcz9" (UID: "dfbccdef-809e-44d4-8970-cfbd4be88d33") : secret "samples-operator-tls" not found
Jul 15 21:02:48.076751 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.076715    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:48.077451 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:48.076837    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 21:02:48.077451 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:48.076849    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8f8c6575-98f4q: secret "image-registry-tls" not found
Jul 15 21:02:48.077451 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:48.076905    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls podName:72565261-8a62-4c4e-9290-ce35e8e1abad nodeName:}" failed. No retries permitted until 2026-07-15 21:02:50.076887436 +0000 UTC m=+39.433806032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls") pod "image-registry-d8f8c6575-98f4q" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad") : secret "image-registry-tls" not found
Jul 15 21:02:48.296569 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.296532    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf"]
Jul 15 21:02:48.300655 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.300619    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf"
Jul 15 21:02:48.305574 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.305548    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tmlgr\""
Jul 15 21:02:48.318542 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.318465    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf"]
Jul 15 21:02:48.367350 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.367267    2580 generic.go:356] "Generic (PLEG): container finished" podID="b1899eb2-d219-4554-910b-e0e59f2b22c2" containerID="73c1a31464d14da50bc9f775ef222be85343ce579e33095c73e9313f0406fc8d" exitCode=0
Jul 15 21:02:48.367350 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.367343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" event={"ID":"b1899eb2-d219-4554-910b-e0e59f2b22c2","Type":"ContainerDied","Data":"73c1a31464d14da50bc9f775ef222be85343ce579e33095c73e9313f0406fc8d"}
Jul 15 21:02:48.380326 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.380293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmmzr\" (UniqueName: \"kubernetes.io/projected/9e35d5fb-3b9a-4029-b97b-fbbf6b45d1e2-kube-api-access-qmmzr\") pod \"network-check-source-869dcfcdcd-hb2sf\" (UID: \"9e35d5fb-3b9a-4029-b97b-fbbf6b45d1e2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf"
Jul 15 21:02:48.481167 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.481119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmmzr\" (UniqueName: \"kubernetes.io/projected/9e35d5fb-3b9a-4029-b97b-fbbf6b45d1e2-kube-api-access-qmmzr\") pod \"network-check-source-869dcfcdcd-hb2sf\" (UID: \"9e35d5fb-3b9a-4029-b97b-fbbf6b45d1e2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf"
Jul 15 21:02:48.490677 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.490645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmmzr\" (UniqueName: \"kubernetes.io/projected/9e35d5fb-3b9a-4029-b97b-fbbf6b45d1e2-kube-api-access-qmmzr\") pod \"network-check-source-869dcfcdcd-hb2sf\" (UID: \"9e35d5fb-3b9a-4029-b97b-fbbf6b45d1e2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf"
Jul 15 21:02:48.613946 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.613899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf"
Jul 15 21:02:48.884316 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:48.884271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:48.884516 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:48.884434    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:02:48.887014 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:48.884975    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls podName:c5bef743-7548-4f72-8caa-6798e573e30b nodeName:}" failed. No retries permitted until 2026-07-15 21:02:52.88472844 +0000 UTC m=+42.241647024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls") pod "dns-default-79q2z" (UID: "c5bef743-7548-4f72-8caa-6798e573e30b") : secret "dns-default-metrics-tls" not found
Jul 15 21:02:49.792360 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:49.792311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:49.793059 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:49.792487    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 21:02:49.793059 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:49.792570    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls podName:dfbccdef-809e-44d4-8970-cfbd4be88d33 nodeName:}" failed. No retries permitted until 2026-07-15 21:02:53.792547206 +0000 UTC m=+43.149465782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls") pod "cluster-samples-operator-65db858894-nhcz9" (UID: "dfbccdef-809e-44d4-8970-cfbd4be88d33") : secret "samples-operator-tls" not found
Jul 15 21:02:50.094428 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:50.094331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:50.094617 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:50.094485    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 21:02:50.094617 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:50.094501    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8f8c6575-98f4q: secret "image-registry-tls" not found
Jul 15 21:02:50.094617 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:50.094569    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls podName:72565261-8a62-4c4e-9290-ce35e8e1abad nodeName:}" failed. No retries permitted until 2026-07-15 21:02:54.094549113 +0000 UTC m=+43.451467691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls") pod "image-registry-d8f8c6575-98f4q" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad") : secret "image-registry-tls" not found
Jul 15 21:02:50.590715 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:50.590683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf"]
Jul 15 21:02:50.595995 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:02:50.595948    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e35d5fb_3b9a_4029_b97b_fbbf6b45d1e2.slice/crio-75b0290122f22f88650888ebb824510e9ced3af873d424d57a2af64a09dce3fa WatchSource:0}: Error finding container 75b0290122f22f88650888ebb824510e9ced3af873d424d57a2af64a09dce3fa: Status 404 returned error can't find the container with id 75b0290122f22f88650888ebb824510e9ced3af873d424d57a2af64a09dce3fa
Jul 15 21:02:51.377082 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.377049    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/0.log"
Jul 15 21:02:51.377568 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.377094    2580 generic.go:356] "Generic (PLEG): container finished" podID="72014806-cd64-45ba-be82-a71c7d7076fb" containerID="11fcb47b3a314ed45a41838350f3e24c34606ee097700e2f176d3e8b18482782" exitCode=255
Jul 15 21:02:51.377568 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.377194    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-mjds2" event={"ID":"72014806-cd64-45ba-be82-a71c7d7076fb","Type":"ContainerDied","Data":"11fcb47b3a314ed45a41838350f3e24c34606ee097700e2f176d3e8b18482782"}
Jul 15 21:02:51.377568 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.377414    2580 scope.go:117] "RemoveContainer" containerID="11fcb47b3a314ed45a41838350f3e24c34606ee097700e2f176d3e8b18482782"
Jul 15 21:02:51.379152 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.379110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l" event={"ID":"1d90ed07-ed6a-458d-956a-1da3260948d2","Type":"ContainerStarted","Data":"c489e21e126debcb6b2104f941858708a9c095a89512f1afab597d68f7e94373"}
Jul 15 21:02:51.380827 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.380798    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jv7fg" event={"ID":"8da41a2e-7aef-4cc5-87a1-05a9570f8642","Type":"ContainerStarted","Data":"5e37a8f57c0135db7f3f0ff683062fa57ab185254ca07f9f90cbeebea08f210a"}
Jul 15 21:02:51.382569 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.382544    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf" event={"ID":"9e35d5fb-3b9a-4029-b97b-fbbf6b45d1e2","Type":"ContainerStarted","Data":"75b0290122f22f88650888ebb824510e9ced3af873d424d57a2af64a09dce3fa"}
Jul 15 21:02:51.386364 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.386341    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" event={"ID":"b1899eb2-d219-4554-910b-e0e59f2b22c2","Type":"ContainerStarted","Data":"3f964cabac801f4fe759307ac0e310a87ce881ce7b958f8145666adfbc43152c"}
Jul 15 21:02:51.417049 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.416947    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jv7fg" podStartSLOduration=2.416834603 podStartE2EDuration="6.416923176s" totalImagesPullingTime="4.000088573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:51.416624202 +0000 UTC m=+40.773542801" watchObservedRunningTime="2026-07-15 21:02:51.416923176 +0000 UTC m=+40.773841774"
Jul 15 21:02:51.452321 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:51.452260    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dsjsm" podStartSLOduration=10.443502743 podStartE2EDuration="40.452238629s" totalImagesPullingTime="30.008735886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:51.450823859 +0000 UTC m=+40.807742456" watchObservedRunningTime="2026-07-15 21:02:51.452238629 +0000 UTC m=+40.809157227"
Jul 15 21:02:52.181788 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.181723    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l" podStartSLOduration=2.482328195 podStartE2EDuration="6.181699345s" totalImagesPullingTime="3.69937115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:51.47313494 +0000 UTC m=+40.830053539" watchObservedRunningTime="2026-07-15 21:02:52.181699345 +0000 UTC m=+41.538617943"
Jul 15 21:02:52.182046 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.182015    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jzqt9"]
Jul 15 21:02:52.206649 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.206556    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jzqt9"]
Jul 15 21:02:52.206830 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.206701    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.210260 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.210003    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 21:02:52.210260 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.210012    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 21:02:52.210260 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.210018    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 21:02:52.210260 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.210106    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 21:02:52.210857 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.210834    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s5tt8\""
Jul 15 21:02:52.317291 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.317257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27177627-91b2-4b20-a8ac-7900de8544ca-crio-socket\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.317486 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.317309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.317486 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.317463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.317604 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.317534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27177627-91b2-4b20-a8ac-7900de8544ca-data-volume\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.317658 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.317629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfbqd\" (UniqueName: \"kubernetes.io/projected/27177627-91b2-4b20-a8ac-7900de8544ca-kube-api-access-cfbqd\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.390904 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.390871    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/1.log"
Jul 15 21:02:52.391438 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.391412    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/0.log"
Jul 15 21:02:52.391554 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.391461    2580 generic.go:356] "Generic (PLEG): container finished" podID="72014806-cd64-45ba-be82-a71c7d7076fb" containerID="324ae603917dc3923fa6309dd150f9e08f8b660da1c3592dcd33292b7181661b" exitCode=255
Jul 15 21:02:52.391627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.391552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-mjds2" event={"ID":"72014806-cd64-45ba-be82-a71c7d7076fb","Type":"ContainerDied","Data":"324ae603917dc3923fa6309dd150f9e08f8b660da1c3592dcd33292b7181661b"}
Jul 15 21:02:52.391627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.391599    2580 scope.go:117] "RemoveContainer" containerID="11fcb47b3a314ed45a41838350f3e24c34606ee097700e2f176d3e8b18482782"
Jul 15 21:02:52.391982 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.391814    2580 scope.go:117] "RemoveContainer" containerID="324ae603917dc3923fa6309dd150f9e08f8b660da1c3592dcd33292b7181661b"
Jul 15 21:02:52.392089 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.392029    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-mjds2_openshift-console-operator(72014806-cd64-45ba-be82-a71c7d7076fb)\"" pod="openshift-console-operator/console-operator-7866497494-mjds2" podUID="72014806-cd64-45ba-be82-a71c7d7076fb"
Jul 15 21:02:52.418792 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.418762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.418946 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.418823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27177627-91b2-4b20-a8ac-7900de8544ca-data-volume\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.418946 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.418860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfbqd\" (UniqueName: \"kubernetes.io/projected/27177627-91b2-4b20-a8ac-7900de8544ca-kube-api-access-cfbqd\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.418946 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.418893    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27177627-91b2-4b20-a8ac-7900de8544ca-crio-socket\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.418946 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.418917    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:52.419189 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.419001    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:02:52.918977207 +0000 UTC m=+42.275895783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:52.419189 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.419069    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 21:02:52.419189 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.418924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.419189 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.419129    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:02:52.919111823 +0000 UTC m=+42.276030415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : configmap "kube-rbac-proxy" not found
Jul 15 21:02:52.419189 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.419174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27177627-91b2-4b20-a8ac-7900de8544ca-crio-socket\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.419370 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.419196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27177627-91b2-4b20-a8ac-7900de8544ca-data-volume\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.433296 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.433236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfbqd\" (UniqueName: \"kubernetes.io/projected/27177627-91b2-4b20-a8ac-7900de8544ca-kube-api-access-cfbqd\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.924587 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.924543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.924841 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.924666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:52.924841 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.924697    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 21:02:52.924841 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.924774    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:02:53.924755023 +0000 UTC m=+43.281673599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : configmap "kube-rbac-proxy" not found
Jul 15 21:02:52.924841 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.924797    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:52.924841 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:52.924707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:02:52.924841 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.924797    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 21:02:52.925136 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.924853    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:02:53.924837076 +0000 UTC m=+43.281755665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:52.925136 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:52.924884    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls podName:c5bef743-7548-4f72-8caa-6798e573e30b nodeName:}" failed. No retries permitted until 2026-07-15 21:03:00.924874545 +0000 UTC m=+50.281793121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls") pod "dns-default-79q2z" (UID: "c5bef743-7548-4f72-8caa-6798e573e30b") : secret "dns-default-metrics-tls" not found
Jul 15 21:02:53.396264 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:53.396234    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/1.log"
Jul 15 21:02:53.396699 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:53.396658    2580 scope.go:117] "RemoveContainer" containerID="324ae603917dc3923fa6309dd150f9e08f8b660da1c3592dcd33292b7181661b"
Jul 15 21:02:53.396879 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:53.396853    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-mjds2_openshift-console-operator(72014806-cd64-45ba-be82-a71c7d7076fb)\"" pod="openshift-console-operator/console-operator-7866497494-mjds2" podUID="72014806-cd64-45ba-be82-a71c7d7076fb"
Jul 15 21:02:53.834152 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:53.832600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:02:53.834152 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:53.832670    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 15 21:02:53.834152 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:53.832754    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls podName:dfbccdef-809e-44d4-8970-cfbd4be88d33 nodeName:}" failed. No retries permitted until 2026-07-15 21:03:01.832734171 +0000 UTC m=+51.189652769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls") pod "cluster-samples-operator-65db858894-nhcz9" (UID: "dfbccdef-809e-44d4-8970-cfbd4be88d33") : secret "samples-operator-tls" not found
Jul 15 21:02:53.933653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:53.933621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:53.933779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:53.933694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:53.933779 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:53.933757    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 21:02:53.933844 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:53.933799    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:53.933844 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:53.933825    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:02:55.933808836 +0000 UTC m=+45.290727410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : configmap "kube-rbac-proxy" not found
Jul 15 21:02:53.933844 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:53.933842    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:02:55.933833628 +0000 UTC m=+45.290752202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:53.974929 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:53.974906    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-599sr_855e91fe-3c9d-4f36-a073-09eecd10ed4c/dns-node-resolver/0.log"
Jul 15 21:02:54.135298 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:54.135203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:02:54.135498 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:54.135362    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 21:02:54.135498 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:54.135382    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8f8c6575-98f4q: secret "image-registry-tls" not found
Jul 15 21:02:54.135498 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:54.135452    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls podName:72565261-8a62-4c4e-9290-ce35e8e1abad nodeName:}" failed. No retries permitted until 2026-07-15 21:03:02.135433521 +0000 UTC m=+51.492352103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls") pod "image-registry-d8f8c6575-98f4q" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad") : secret "image-registry-tls" not found
Jul 15 21:02:54.400156 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:54.400070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf" event={"ID":"9e35d5fb-3b9a-4029-b97b-fbbf6b45d1e2","Type":"ContainerStarted","Data":"2615e265f90c561bb436ed29575718f3338567533373160da3b2b6ab9ab51a81"}
Jul 15 21:02:54.426836 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:54.426785    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-hb2sf" podStartSLOduration=3.110718691 podStartE2EDuration="6.426769112s" totalImagesPullingTime="3.316050421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:02:54.425008957 +0000 UTC m=+43.781927555" watchObservedRunningTime="2026-07-15 21:02:54.426769112 +0000 UTC m=+43.783687708"
Jul 15 21:02:55.175256 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:55.175224    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lc5h4_0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e/node-ca/0.log"
Jul 15 21:02:55.950566 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:55.950523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:55.950988 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:55.950624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:55.950988 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:55.950669    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:55.950988 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:55.950699    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 21:02:55.950988 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:55.950735    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:02:59.950720634 +0000 UTC m=+49.307639208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:55.950988 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:55.950750    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:02:59.950743842 +0000 UTC m=+49.307662417 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : configmap "kube-rbac-proxy" not found
Jul 15 21:02:56.184794 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:56.184762    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jv7fg_8da41a2e-7aef-4cc5-87a1-05a9570f8642/serve-healthcheck-canary/0.log"
Jul 15 21:02:56.467796 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:56.467761    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:56.467978 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:56.467822    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:02:56.468187 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:56.468173    2580 scope.go:117] "RemoveContainer" containerID="324ae603917dc3923fa6309dd150f9e08f8b660da1c3592dcd33292b7181661b"
Jul 15 21:02:56.468339 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:56.468322    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-mjds2_openshift-console-operator(72014806-cd64-45ba-be82-a71c7d7076fb)\"" pod="openshift-console-operator/console-operator-7866497494-mjds2" podUID="72014806-cd64-45ba-be82-a71c7d7076fb"
Jul 15 21:02:59.983789 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:59.983731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:59.984293 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:02:59.983837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:02:59.984293 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:59.983902    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:59.984293 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:59.983908    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 21:02:59.984293 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:59.984013    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:03:07.983991079 +0000 UTC m=+57.340909672 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : secret "insights-runtime-extractor-tls" not found
Jul 15 21:02:59.984293 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:02:59.984046    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm podName:27177627-91b2-4b20-a8ac-7900de8544ca nodeName:}" failed. No retries permitted until 2026-07-15 21:03:07.984033999 +0000 UTC m=+57.340952574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqt9" (UID: "27177627-91b2-4b20-a8ac-7900de8544ca") : configmap "kube-rbac-proxy" not found
Jul 15 21:03:00.993323 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:00.993282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:03:00.995637 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:00.995617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5bef743-7548-4f72-8caa-6798e573e30b-metrics-tls\") pod \"dns-default-79q2z\" (UID: \"c5bef743-7548-4f72-8caa-6798e573e30b\") " pod="openshift-dns/dns-default-79q2z"
Jul 15 21:03:01.295113 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:01.295076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-79q2z"
Jul 15 21:03:01.443718 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:01.443687    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-79q2z"]
Jul 15 21:03:01.446694 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:01.446662    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5bef743_7548_4f72_8caa_6798e573e30b.slice/crio-f58c7fbf1a3f93dc076e1ef01645f5a170c29e72fa72f8109b589f5c1e3fb6c3 WatchSource:0}: Error finding container f58c7fbf1a3f93dc076e1ef01645f5a170c29e72fa72f8109b589f5c1e3fb6c3: Status 404 returned error can't find the container with id f58c7fbf1a3f93dc076e1ef01645f5a170c29e72fa72f8109b589f5c1e3fb6c3
Jul 15 21:03:01.900450 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:01.900411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:03:01.902734 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:01.902708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/dfbccdef-809e-44d4-8970-cfbd4be88d33-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nhcz9\" (UID: \"dfbccdef-809e-44d4-8970-cfbd4be88d33\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:03:01.962220 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:01.962179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"
Jul 15 21:03:02.103159 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:02.103090    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9"]
Jul 15 21:03:02.203297 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:02.203262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:03:02.205610 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:02.205583    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") pod \"image-registry-d8f8c6575-98f4q\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") " pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:03:02.221488 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:02.221461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:03:02.367904 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:02.367872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8f8c6575-98f4q"]
Jul 15 21:03:02.371180 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:02.371144    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72565261_8a62_4c4e_9290_ce35e8e1abad.slice/crio-5ea95bd1ba6d547cae595b3a44b17960071f1a289a59c881a9f1f69c5dea55c3 WatchSource:0}: Error finding container 5ea95bd1ba6d547cae595b3a44b17960071f1a289a59c881a9f1f69c5dea55c3: Status 404 returned error can't find the container with id 5ea95bd1ba6d547cae595b3a44b17960071f1a289a59c881a9f1f69c5dea55c3
Jul 15 21:03:02.420670 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:02.420619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79q2z" event={"ID":"c5bef743-7548-4f72-8caa-6798e573e30b","Type":"ContainerStarted","Data":"f58c7fbf1a3f93dc076e1ef01645f5a170c29e72fa72f8109b589f5c1e3fb6c3"}
Jul 15 21:03:02.421951 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:02.421924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q" event={"ID":"72565261-8a62-4c4e-9290-ce35e8e1abad","Type":"ContainerStarted","Data":"5ea95bd1ba6d547cae595b3a44b17960071f1a289a59c881a9f1f69c5dea55c3"}
Jul 15 21:03:02.423169 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:02.423140    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9" event={"ID":"dfbccdef-809e-44d4-8970-cfbd4be88d33","Type":"ContainerStarted","Data":"d6c522d98de6c5fde9667db00ba7baf8d5bb119e8fcbfccf577b90e8259bace2"}
Jul 15 21:03:03.431741 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:03.431707    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q" event={"ID":"72565261-8a62-4c4e-9290-ce35e8e1abad","Type":"ContainerStarted","Data":"3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab"}
Jul 15 21:03:03.432242 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:03.431837    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:03:03.455541 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:03.455496    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q" podStartSLOduration=17.455482274 podStartE2EDuration="17.455482274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:03.454732061 +0000 UTC m=+52.811650658" watchObservedRunningTime="2026-07-15 21:03:03.455482274 +0000 UTC m=+52.812400848"
Jul 15 21:03:04.438108 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:04.438010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9" event={"ID":"dfbccdef-809e-44d4-8970-cfbd4be88d33","Type":"ContainerStarted","Data":"dd72040f0a5d4a045a477e7c9306c175f9d243b8e3d61d850e88e7fded824899"}
Jul 15 21:03:04.438108 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:04.438071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9" event={"ID":"dfbccdef-809e-44d4-8970-cfbd4be88d33","Type":"ContainerStarted","Data":"629c0ce245f850699affa0db124a6ced26e4550ec9a0e79e4ae10676d7a28f62"}
Jul 15 21:03:04.439673 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:04.439650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79q2z" event={"ID":"c5bef743-7548-4f72-8caa-6798e573e30b","Type":"ContainerStarted","Data":"b722d6c08b34fdf12165ea6cbc5c8d661c3c5adb22a5e1ac52d82013779f2204"}
Jul 15 21:03:05.445372 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:05.445332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79q2z" event={"ID":"c5bef743-7548-4f72-8caa-6798e573e30b","Type":"ContainerStarted","Data":"d370f3a670342840a4dbb21b3db32e8ae1e6d8f728e04682478499528199f3d8"}
Jul 15 21:03:05.466061 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:05.466004    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-79q2z" podStartSLOduration=17.627617396 podStartE2EDuration="20.465983567s" totalImagesPullingTime="2.838366171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:05.46469164 +0000 UTC m=+54.821610235" watchObservedRunningTime="2026-07-15 21:03:05.465983567 +0000 UTC m=+54.822902159"
Jul 15 21:03:05.466320 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:05.466294    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nhcz9" podStartSLOduration=17.33905451 podStartE2EDuration="19.466286513s" totalImagesPullingTime="2.127232003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:04.45858416 +0000 UTC m=+53.815502751" watchObservedRunningTime="2026-07-15 21:03:05.466286513 +0000 UTC m=+54.823205104"
Jul 15 21:03:06.448666 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:06.448633    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-79q2z"
Jul 15 21:03:07.128887 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:07.128858    2580 scope.go:117] "RemoveContainer" containerID="324ae603917dc3923fa6309dd150f9e08f8b660da1c3592dcd33292b7181661b"
Jul 15 21:03:07.453917 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:07.453840    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:03:07.454303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:07.454213    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/1.log"
Jul 15 21:03:07.454303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:07.454246    2580 generic.go:356] "Generic (PLEG): container finished" podID="72014806-cd64-45ba-be82-a71c7d7076fb" containerID="e85bfe2b999ed5692f33dfdd91ec80b2fe752d42c0543552dcec14f76e0d9e0e" exitCode=255
Jul 15 21:03:07.454373 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:07.454322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-mjds2" event={"ID":"72014806-cd64-45ba-be82-a71c7d7076fb","Type":"ContainerDied","Data":"e85bfe2b999ed5692f33dfdd91ec80b2fe752d42c0543552dcec14f76e0d9e0e"}
Jul 15 21:03:07.454421 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:07.454377    2580 scope.go:117] "RemoveContainer" containerID="324ae603917dc3923fa6309dd150f9e08f8b660da1c3592dcd33292b7181661b"
Jul 15 21:03:07.454771 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:07.454746    2580 scope.go:117] "RemoveContainer" containerID="e85bfe2b999ed5692f33dfdd91ec80b2fe752d42c0543552dcec14f76e0d9e0e"
Jul 15 21:03:07.454935 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:03:07.454915    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-mjds2_openshift-console-operator(72014806-cd64-45ba-be82-a71c7d7076fb)\"" pod="openshift-console-operator/console-operator-7866497494-mjds2" podUID="72014806-cd64-45ba-be82-a71c7d7076fb"
Jul 15 21:03:08.051814 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:08.051759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:03:08.052063 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:08.051841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:03:08.052400 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:08.052374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27177627-91b2-4b20-a8ac-7900de8544ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:03:08.054071 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:08.054054    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27177627-91b2-4b20-a8ac-7900de8544ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqt9\" (UID: \"27177627-91b2-4b20-a8ac-7900de8544ca\") " pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:03:08.117967 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:08.117923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jzqt9"
Jul 15 21:03:08.263500 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:08.263469    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jzqt9"]
Jul 15 21:03:08.266198 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:08.266168    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27177627_91b2_4b20_a8ac_7900de8544ca.slice/crio-5943da42ee3862695af345eea0c4c39b86ea1ea52ea938403b883c5736af0909 WatchSource:0}: Error finding container 5943da42ee3862695af345eea0c4c39b86ea1ea52ea938403b883c5736af0909: Status 404 returned error can't find the container with id 5943da42ee3862695af345eea0c4c39b86ea1ea52ea938403b883c5736af0909
Jul 15 21:03:08.458630 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:08.458526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzqt9" event={"ID":"27177627-91b2-4b20-a8ac-7900de8544ca","Type":"ContainerStarted","Data":"5943da42ee3862695af345eea0c4c39b86ea1ea52ea938403b883c5736af0909"}
Jul 15 21:03:08.460004 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:08.459984    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:03:10.352602 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:10.352574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gckpk"
Jul 15 21:03:13.488682 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:13.488587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzqt9" event={"ID":"27177627-91b2-4b20-a8ac-7900de8544ca","Type":"ContainerStarted","Data":"f6419c0b0e850ee406d93a3056494f774f9f74ae3424393381542e7c7851bd1d"}
Jul 15 21:03:15.822208 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:15.822164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:03:15.824874 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:15.824848    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 21:03:15.835235 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:15.835208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e184053-7574-434b-8873-db1b343c9589-original-pull-secret\") pod \"global-pull-secret-syncer-6ldsz\" (UID: \"4e184053-7574-434b-8873-db1b343c9589\") " pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:03:15.922925 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:15.922886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:03:15.925704 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:15.925679    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 21:03:15.936154 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:15.936131    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3220222-dfc1-4a57-a205-809e0fba860e-metrics-certs\") pod \"network-metrics-daemon-xr98b\" (UID: \"e3220222-dfc1-4a57-a205-809e0fba860e\") " pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:03:16.024389 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.024346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:03:16.027133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.027106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm9j9\" (UniqueName: \"kubernetes.io/projected/31bad667-66ee-41eb-9643-a72135c77989-kube-api-access-nm9j9\") pod \"network-check-target-kmgwk\" (UID: \"31bad667-66ee-41eb-9643-a72135c77989\") " pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:03:16.045602 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.045575    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rwswm\""
Jul 15 21:03:16.049497 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.049468    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ldsz"
Jul 15 21:03:16.052607 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.052576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xr98b"
Jul 15 21:03:16.058045 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.057676    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfzc2\""
Jul 15 21:03:16.065936 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.065909    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:03:16.414645 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.414601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kmgwk"]
Jul 15 21:03:16.418352 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:16.418328    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31bad667_66ee_41eb_9643_a72135c77989.slice/crio-18349e5dba9e9fbbeb034626496b16f0dad47853a06ea0592387deb8543b519f WatchSource:0}: Error finding container 18349e5dba9e9fbbeb034626496b16f0dad47853a06ea0592387deb8543b519f: Status 404 returned error can't find the container with id 18349e5dba9e9fbbeb034626496b16f0dad47853a06ea0592387deb8543b519f
Jul 15 21:03:16.456345 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.456324    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-79q2z"
Jul 15 21:03:16.467192 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.467163    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:03:16.467292 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.467206    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:03:16.467484 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.467473    2580 scope.go:117] "RemoveContainer" containerID="e85bfe2b999ed5692f33dfdd91ec80b2fe752d42c0543552dcec14f76e0d9e0e"
Jul 15 21:03:16.467610 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:03:16.467594    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-mjds2_openshift-console-operator(72014806-cd64-45ba-be82-a71c7d7076fb)\"" pod="openshift-console-operator/console-operator-7866497494-mjds2" podUID="72014806-cd64-45ba-be82-a71c7d7076fb"
Jul 15 21:03:16.501345 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.501313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzqt9" event={"ID":"27177627-91b2-4b20-a8ac-7900de8544ca","Type":"ContainerStarted","Data":"c51ba4e05615f6062f11c8e6de50cb4b3628f861fbf3d0bb271357e36fdf7295"}
Jul 15 21:03:16.502720 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.502695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kmgwk" event={"ID":"31bad667-66ee-41eb-9643-a72135c77989","Type":"ContainerStarted","Data":"60a5eb44ae7ee7eaaeb1c8a21deca772f9ea5c6a7576a4174bd2e7a669c495ca"}
Jul 15 21:03:16.502823 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.502731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kmgwk" event={"ID":"31bad667-66ee-41eb-9643-a72135c77989","Type":"ContainerStarted","Data":"18349e5dba9e9fbbeb034626496b16f0dad47853a06ea0592387deb8543b519f"}
Jul 15 21:03:16.502868 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.502829    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:03:16.576999 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.576930    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kmgwk" podStartSLOduration=65.576914538 podStartE2EDuration="1m5.576914538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:16.572466452 +0000 UTC m=+65.929385048" watchObservedRunningTime="2026-07-15 21:03:16.576914538 +0000 UTC m=+65.933833240"
Jul 15 21:03:16.610906 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.610870    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6ldsz"]
Jul 15 21:03:16.611826 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:16.611796    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3220222_dfc1_4a57_a205_809e0fba860e.slice/crio-3d8811f025d2a83b02a30786192b15b303a2e9ea1476fc4a9ff3f56565f20f52 WatchSource:0}: Error finding container 3d8811f025d2a83b02a30786192b15b303a2e9ea1476fc4a9ff3f56565f20f52: Status 404 returned error can't find the container with id 3d8811f025d2a83b02a30786192b15b303a2e9ea1476fc4a9ff3f56565f20f52
Jul 15 21:03:16.612056 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:16.612033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e184053_7574_434b_8873_db1b343c9589.slice/crio-61786181b9e3d472c0b6988514a71626a06bf4944f965ecb96d9bc4af5c494c3 WatchSource:0}: Error finding container 61786181b9e3d472c0b6988514a71626a06bf4944f965ecb96d9bc4af5c494c3: Status 404 returned error can't find the container with id 61786181b9e3d472c0b6988514a71626a06bf4944f965ecb96d9bc4af5c494c3
Jul 15 21:03:16.612196 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.612126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xr98b"]
Jul 15 21:03:16.706403 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.706372    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"]
Jul 15 21:03:16.718550 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.718484    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"
Jul 15 21:03:16.721927 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.721908    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-s6zhd\""
Jul 15 21:03:16.722417 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.722402    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 15 21:03:16.722754 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.722737    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 15 21:03:16.729181 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.729161    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"]
Jul 15 21:03:16.757275 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.757246    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"]
Jul 15 21:03:16.777695 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.777661    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d8f8c6575-98f4q"]
Jul 15 21:03:16.777841 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.777776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:16.782365 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.782343    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 21:03:16.782662 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.782614    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 21:03:16.782785 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.782770    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 21:03:16.783006 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.782993    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 15 21:03:16.826592 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.826563    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"]
Jul 15 21:03:16.830805 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.830775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/601f2a97-4158-4d1b-859b-2bfca7738159-klusterlet-config\") pod \"klusterlet-addon-workmgr-766858c957-9z56t\" (UID: \"601f2a97-4158-4d1b-859b-2bfca7738159\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:16.830932 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.830821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02992d82-53fc-4b51-9da7-0f59ecdce970-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tq7ds\" (UID: \"02992d82-53fc-4b51-9da7-0f59ecdce970\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"
Jul 15 21:03:16.830932 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.830895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/601f2a97-4158-4d1b-859b-2bfca7738159-tmp\") pod \"klusterlet-addon-workmgr-766858c957-9z56t\" (UID: \"601f2a97-4158-4d1b-859b-2bfca7738159\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:16.831075 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.830925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgp5t\" (UniqueName: \"kubernetes.io/projected/601f2a97-4158-4d1b-859b-2bfca7738159-kube-api-access-dgp5t\") pod \"klusterlet-addon-workmgr-766858c957-9z56t\" (UID: \"601f2a97-4158-4d1b-859b-2bfca7738159\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:16.831075 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.830981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02992d82-53fc-4b51-9da7-0f59ecdce970-nginx-conf\") pod \"networking-console-plugin-6cf856758c-tq7ds\" (UID: \"02992d82-53fc-4b51-9da7-0f59ecdce970\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"
Jul 15 21:03:16.871487 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.871453    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85bc5987-9hdxw"]
Jul 15 21:03:16.889758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.889731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:16.907464 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.907428    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85bc5987-9hdxw"]
Jul 15 21:03:16.932103 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.932074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/601f2a97-4158-4d1b-859b-2bfca7738159-klusterlet-config\") pod \"klusterlet-addon-workmgr-766858c957-9z56t\" (UID: \"601f2a97-4158-4d1b-859b-2bfca7738159\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:16.932276 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.932120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02992d82-53fc-4b51-9da7-0f59ecdce970-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tq7ds\" (UID: \"02992d82-53fc-4b51-9da7-0f59ecdce970\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"
Jul 15 21:03:16.932276 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.932176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/601f2a97-4158-4d1b-859b-2bfca7738159-tmp\") pod \"klusterlet-addon-workmgr-766858c957-9z56t\" (UID: \"601f2a97-4158-4d1b-859b-2bfca7738159\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:16.932276 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.932200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgp5t\" (UniqueName: \"kubernetes.io/projected/601f2a97-4158-4d1b-859b-2bfca7738159-kube-api-access-dgp5t\") pod \"klusterlet-addon-workmgr-766858c957-9z56t\" (UID: \"601f2a97-4158-4d1b-859b-2bfca7738159\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:16.932444 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.932394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02992d82-53fc-4b51-9da7-0f59ecdce970-nginx-conf\") pod \"networking-console-plugin-6cf856758c-tq7ds\" (UID: \"02992d82-53fc-4b51-9da7-0f59ecdce970\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"
Jul 15 21:03:16.932671 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.932637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/601f2a97-4158-4d1b-859b-2bfca7738159-tmp\") pod \"klusterlet-addon-workmgr-766858c957-9z56t\" (UID: \"601f2a97-4158-4d1b-859b-2bfca7738159\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:16.933219 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.933190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02992d82-53fc-4b51-9da7-0f59ecdce970-nginx-conf\") pod \"networking-console-plugin-6cf856758c-tq7ds\" (UID: \"02992d82-53fc-4b51-9da7-0f59ecdce970\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"
Jul 15 21:03:16.934854 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.934833    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02992d82-53fc-4b51-9da7-0f59ecdce970-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tq7ds\" (UID: \"02992d82-53fc-4b51-9da7-0f59ecdce970\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"
Jul 15 21:03:16.935031 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.935010    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/601f2a97-4158-4d1b-859b-2bfca7738159-klusterlet-config\") pod \"klusterlet-addon-workmgr-766858c957-9z56t\" (UID: \"601f2a97-4158-4d1b-859b-2bfca7738159\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:16.958790 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:16.958734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgp5t\" (UniqueName: \"kubernetes.io/projected/601f2a97-4158-4d1b-859b-2bfca7738159-kube-api-access-dgp5t\") pod \"klusterlet-addon-workmgr-766858c957-9z56t\" (UID: \"601f2a97-4158-4d1b-859b-2bfca7738159\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:17.027517 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.027466    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"
Jul 15 21:03:17.032792 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.032754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8m2m\" (UniqueName: \"kubernetes.io/projected/158e1146-c430-43a7-8217-6bec05abcf73-kube-api-access-x8m2m\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.032927 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.032800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/158e1146-c430-43a7-8217-6bec05abcf73-ca-trust-extracted\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.032927 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.032866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/158e1146-c430-43a7-8217-6bec05abcf73-image-registry-private-configuration\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.032927 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.032911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/158e1146-c430-43a7-8217-6bec05abcf73-registry-certificates\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.033078 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.032936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/158e1146-c430-43a7-8217-6bec05abcf73-trusted-ca\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.033078 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.033031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/158e1146-c430-43a7-8217-6bec05abcf73-registry-tls\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.033078 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.033055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/158e1146-c430-43a7-8217-6bec05abcf73-bound-sa-token\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.033187 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.033087    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/158e1146-c430-43a7-8217-6bec05abcf73-installation-pull-secrets\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.088133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.088104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:17.133509 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.133480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/158e1146-c430-43a7-8217-6bec05abcf73-registry-tls\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.133644 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.133522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/158e1146-c430-43a7-8217-6bec05abcf73-bound-sa-token\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.133644 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.133563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/158e1146-c430-43a7-8217-6bec05abcf73-installation-pull-secrets\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.133644 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.133629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8m2m\" (UniqueName: \"kubernetes.io/projected/158e1146-c430-43a7-8217-6bec05abcf73-kube-api-access-x8m2m\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.133819 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.133674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/158e1146-c430-43a7-8217-6bec05abcf73-ca-trust-extracted\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.133819 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.133707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/158e1146-c430-43a7-8217-6bec05abcf73-image-registry-private-configuration\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.133819 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.133745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/158e1146-c430-43a7-8217-6bec05abcf73-registry-certificates\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.133819 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.133774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/158e1146-c430-43a7-8217-6bec05abcf73-trusted-ca\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.134689 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.134611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/158e1146-c430-43a7-8217-6bec05abcf73-registry-certificates\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.134689 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.134669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/158e1146-c430-43a7-8217-6bec05abcf73-ca-trust-extracted\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.134855 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.134748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/158e1146-c430-43a7-8217-6bec05abcf73-trusted-ca\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.137144 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.137124    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/158e1146-c430-43a7-8217-6bec05abcf73-registry-tls\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.137260 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.137189    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/158e1146-c430-43a7-8217-6bec05abcf73-installation-pull-secrets\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.139429 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.139408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/158e1146-c430-43a7-8217-6bec05abcf73-image-registry-private-configuration\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.160012 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.159947    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8m2m\" (UniqueName: \"kubernetes.io/projected/158e1146-c430-43a7-8217-6bec05abcf73-kube-api-access-x8m2m\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.165290 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.165236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/158e1146-c430-43a7-8217-6bec05abcf73-bound-sa-token\") pod \"image-registry-85bc5987-9hdxw\" (UID: \"158e1146-c430-43a7-8217-6bec05abcf73\") " pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.201455 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.201216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:17.252850 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.252767    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-tq7ds"]
Jul 15 21:03:17.256433 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:17.256401    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02992d82_53fc_4b51_9da7_0f59ecdce970.slice/crio-ee4b763416eded0a0bceeeff977bdfa3dec48e90cfb6844ddf84ceb89113f71b WatchSource:0}: Error finding container ee4b763416eded0a0bceeeff977bdfa3dec48e90cfb6844ddf84ceb89113f71b: Status 404 returned error can't find the container with id ee4b763416eded0a0bceeeff977bdfa3dec48e90cfb6844ddf84ceb89113f71b
Jul 15 21:03:17.328203 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.327432    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"]
Jul 15 21:03:17.331570 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:17.331536    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601f2a97_4158_4d1b_859b_2bfca7738159.slice/crio-5d046a3954d154a95068337b5dfdfabec7ea041a6e2a6225f2de588a4f0d7daa WatchSource:0}: Error finding container 5d046a3954d154a95068337b5dfdfabec7ea041a6e2a6225f2de588a4f0d7daa: Status 404 returned error can't find the container with id 5d046a3954d154a95068337b5dfdfabec7ea041a6e2a6225f2de588a4f0d7daa
Jul 15 21:03:17.423977 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.423932    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85bc5987-9hdxw"]
Jul 15 21:03:17.428417 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:17.428375    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158e1146_c430_43a7_8217_6bec05abcf73.slice/crio-d80f95049a122e9c042540ece7540efad830400d5e6717336a538298c9eca59d WatchSource:0}: Error finding container d80f95049a122e9c042540ece7540efad830400d5e6717336a538298c9eca59d: Status 404 returned error can't find the container with id d80f95049a122e9c042540ece7540efad830400d5e6717336a538298c9eca59d
Jul 15 21:03:17.508681 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.508649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85bc5987-9hdxw" event={"ID":"158e1146-c430-43a7-8217-6bec05abcf73","Type":"ContainerStarted","Data":"d80f95049a122e9c042540ece7540efad830400d5e6717336a538298c9eca59d"}
Jul 15 21:03:17.510641 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.510580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t" event={"ID":"601f2a97-4158-4d1b-859b-2bfca7738159","Type":"ContainerStarted","Data":"5d046a3954d154a95068337b5dfdfabec7ea041a6e2a6225f2de588a4f0d7daa"}
Jul 15 21:03:17.512247 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.512218    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds" event={"ID":"02992d82-53fc-4b51-9da7-0f59ecdce970","Type":"ContainerStarted","Data":"ee4b763416eded0a0bceeeff977bdfa3dec48e90cfb6844ddf84ceb89113f71b"}
Jul 15 21:03:17.513837 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.513773    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6ldsz" event={"ID":"4e184053-7574-434b-8873-db1b343c9589","Type":"ContainerStarted","Data":"61786181b9e3d472c0b6988514a71626a06bf4944f965ecb96d9bc4af5c494c3"}
Jul 15 21:03:17.515401 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:17.515378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xr98b" event={"ID":"e3220222-dfc1-4a57-a205-809e0fba860e","Type":"ContainerStarted","Data":"3d8811f025d2a83b02a30786192b15b303a2e9ea1476fc4a9ff3f56565f20f52"}
Jul 15 21:03:18.524108 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:18.523757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85bc5987-9hdxw" event={"ID":"158e1146-c430-43a7-8217-6bec05abcf73","Type":"ContainerStarted","Data":"e8e108fcd3b9748b4a6440cce1db99d8c0a262f89740a8d34c2ad5e02bd6c1b9"}
Jul 15 21:03:18.524108 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:18.523934    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:18.529453 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:18.529409    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xr98b" event={"ID":"e3220222-dfc1-4a57-a205-809e0fba860e","Type":"ContainerStarted","Data":"e50c751eb2b0ee1771e5b64d0558df016a0d71a5b15797d4ac8d627be1274454"}
Jul 15 21:03:18.552445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:18.552231    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85bc5987-9hdxw" podStartSLOduration=2.552206528 podStartE2EDuration="2.552206528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:18.549689355 +0000 UTC m=+67.906607951" watchObservedRunningTime="2026-07-15 21:03:18.552206528 +0000 UTC m=+67.909125123"
Jul 15 21:03:19.538380 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:19.538062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xr98b" event={"ID":"e3220222-dfc1-4a57-a205-809e0fba860e","Type":"ContainerStarted","Data":"950b9df887c21c5718369a1d4f7fbfd3b658eace5584e43fefe6b12e691cdb51"}
Jul 15 21:03:19.565673 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:19.565614    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xr98b" podStartSLOduration=66.845512472 podStartE2EDuration="1m8.565591808s" totalImagesPullingTime="1.720079336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:19.565036121 +0000 UTC m=+68.921954718" watchObservedRunningTime="2026-07-15 21:03:19.565591808 +0000 UTC m=+68.922510404"
Jul 15 21:03:25.561868 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.561829    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t" event={"ID":"601f2a97-4158-4d1b-859b-2bfca7738159","Type":"ContainerStarted","Data":"50a0b984883ea7e718c4d0d037c8aa12d0d0ceabd5ce67938369d977b6a51248"}
Jul 15 21:03:25.562381 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.561978    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:25.563416 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.563385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds" event={"ID":"02992d82-53fc-4b51-9da7-0f59ecdce970","Type":"ContainerStarted","Data":"3cefd74c4d2967757926a8eebc787883ea873037fd1ca2c40a7dc20234e7d848"}
Jul 15 21:03:25.564057 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.564037    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t"
Jul 15 21:03:25.564850 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.564826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6ldsz" event={"ID":"4e184053-7574-434b-8873-db1b343c9589","Type":"ContainerStarted","Data":"a74f6467c245d1f4760bffa04bdcfba8269293b06caccba66f879f319c05eaa6"}
Jul 15 21:03:25.566846 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.566827    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzqt9" event={"ID":"27177627-91b2-4b20-a8ac-7900de8544ca","Type":"ContainerStarted","Data":"d17829ccab91618b18c1da6474791056fffe306ff4cd1809ab6920923441a8a4"}
Jul 15 21:03:25.581372 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.581318    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-766858c957-9z56t" podStartSLOduration=2.305992785 podStartE2EDuration="9.581305397s" totalImagesPullingTime="7.275312612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:25.57967656 +0000 UTC m=+74.936595251" watchObservedRunningTime="2026-07-15 21:03:25.581305397 +0000 UTC m=+74.938223993"
Jul 15 21:03:25.596439 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.596402    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6ldsz" podStartSLOduration=66.610411122 podStartE2EDuration="1m14.596389053s" totalImagesPullingTime="7.985977931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:25.595896472 +0000 UTC m=+74.952815068" watchObservedRunningTime="2026-07-15 21:03:25.596389053 +0000 UTC m=+74.953307648"
Jul 15 21:03:25.623081 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.623036    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jzqt9" podStartSLOduration=17.441712764 podStartE2EDuration="33.623022951s" totalImagesPullingTime="16.181310187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:02:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:25.622619536 +0000 UTC m=+74.979538132" watchObservedRunningTime="2026-07-15 21:03:25.623022951 +0000 UTC m=+74.979941546"
Jul 15 21:03:25.639735 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:25.639693    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-tq7ds" podStartSLOduration=2.311412616 podStartE2EDuration="9.639680944s" totalImagesPullingTime="7.328268328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:25.639522328 +0000 UTC m=+74.996440925" watchObservedRunningTime="2026-07-15 21:03:25.639680944 +0000 UTC m=+74.996599540"
Jul 15 21:03:26.783722 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:26.783694    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:03:29.128469 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:29.128437    2580 scope.go:117] "RemoveContainer" containerID="e85bfe2b999ed5692f33dfdd91ec80b2fe752d42c0543552dcec14f76e0d9e0e"
Jul 15 21:03:29.583672 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:29.583639    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:03:29.583842 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:29.583743    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-mjds2" event={"ID":"72014806-cd64-45ba-be82-a71c7d7076fb","Type":"ContainerStarted","Data":"ecfe2f5d2d43fae97d035e89c8ab14a483e10c10af555f10854bf2e88ad9d655"}
Jul 15 21:03:29.584040 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:29.584012    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:03:30.262110 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:30.262078    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-mjds2"
Jul 15 21:03:30.289446 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:30.289395    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-mjds2" podStartSLOduration=40.436358096 podStartE2EDuration="44.28937419s" totalImagesPullingTime="3.853016094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:29.617341441 +0000 UTC m=+78.974260037" watchObservedRunningTime="2026-07-15 21:03:30.28937419 +0000 UTC m=+79.646292808"
Jul 15 21:03:31.429675 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.429645    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4z7rk"]
Jul 15 21:03:31.432425 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.432409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.436351 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.436327    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 21:03:31.436467 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.436384    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 21:03:31.436467 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.436406    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lhkbt\""
Jul 15 21:03:31.436586 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.436536    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 21:03:31.436795 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.436782    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 21:03:31.436971 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.436943    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 21:03:31.437175 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.437161    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 21:03:31.457070 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.457046    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-7gk99"]
Jul 15 21:03:31.460008 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.459988    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.464462 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.464443    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 15 21:03:31.464462 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.464452    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 15 21:03:31.464588 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.464562    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 15 21:03:31.465731 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.465703    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gp6xl\""
Jul 15 21:03:31.486760 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.486733    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-7gk99"]
Jul 15 21:03:31.549279 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fgnp\" (UniqueName: \"kubernetes.io/projected/254e79cf-8922-4bf5-a354-5f08af660818-kube-api-access-8fgnp\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.549437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/254e79cf-8922-4bf5-a354-5f08af660818-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.549437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/254e79cf-8922-4bf5-a354-5f08af660818-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.549437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfd45\" (UniqueName: \"kubernetes.io/projected/45041402-dc9b-404d-a815-99e68f9f1e21-kube-api-access-qfd45\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.549437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.549437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-textfile\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.549437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45041402-dc9b-404d-a815-99e68f9f1e21-metrics-client-ca\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.549623 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45041402-dc9b-404d-a815-99e68f9f1e21-root\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.549623 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-wtmp\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.549623 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/254e79cf-8922-4bf5-a354-5f08af660818-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.549623 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549602    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45041402-dc9b-404d-a815-99e68f9f1e21-sys\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.549623 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-tls\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.549760 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-accelerators-collector-config\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.549760 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/254e79cf-8922-4bf5-a354-5f08af660818-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.549760 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.549675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/254e79cf-8922-4bf5-a354-5f08af660818-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.650519 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45041402-dc9b-404d-a815-99e68f9f1e21-root\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.650519 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-wtmp\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/254e79cf-8922-4bf5-a354-5f08af660818-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45041402-dc9b-404d-a815-99e68f9f1e21-sys\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-tls\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-accelerators-collector-config\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45041402-dc9b-404d-a815-99e68f9f1e21-root\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/254e79cf-8922-4bf5-a354-5f08af660818-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45041402-dc9b-404d-a815-99e68f9f1e21-sys\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/254e79cf-8922-4bf5-a354-5f08af660818-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-wtmp\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fgnp\" (UniqueName: \"kubernetes.io/projected/254e79cf-8922-4bf5-a354-5f08af660818-kube-api-access-8fgnp\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.650766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/254e79cf-8922-4bf5-a354-5f08af660818-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.651281 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/254e79cf-8922-4bf5-a354-5f08af660818-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.651323 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.651267    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/254e79cf-8922-4bf5-a354-5f08af660818-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.651353 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.650914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfd45\" (UniqueName: \"kubernetes.io/projected/45041402-dc9b-404d-a815-99e68f9f1e21-kube-api-access-qfd45\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.651401 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.651375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/254e79cf-8922-4bf5-a354-5f08af660818-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.651445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.651422    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-accelerators-collector-config\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.651987 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.651729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/254e79cf-8922-4bf5-a354-5f08af660818-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.652372 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.652347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.652471 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.652404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-textfile\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.652471 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.652461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45041402-dc9b-404d-a815-99e68f9f1e21-metrics-client-ca\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.653115 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.653094    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45041402-dc9b-404d-a815-99e68f9f1e21-metrics-client-ca\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.653280 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.653256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-textfile\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.653513 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.653486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-tls\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.653713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.653690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/254e79cf-8922-4bf5-a354-5f08af660818-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.654527 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.654499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/254e79cf-8922-4bf5-a354-5f08af660818-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.657866 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.657840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45041402-dc9b-404d-a815-99e68f9f1e21-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.663166 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.663141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fgnp\" (UniqueName: \"kubernetes.io/projected/254e79cf-8922-4bf5-a354-5f08af660818-kube-api-access-8fgnp\") pod \"kube-state-metrics-785778ff74-7gk99\" (UID: \"254e79cf-8922-4bf5-a354-5f08af660818\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.663729 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.663710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfd45\" (UniqueName: \"kubernetes.io/projected/45041402-dc9b-404d-a815-99e68f9f1e21-kube-api-access-qfd45\") pod \"node-exporter-4z7rk\" (UID: \"45041402-dc9b-404d-a815-99e68f9f1e21\") " pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.742066 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.741917    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4z7rk"
Jul 15 21:03:31.754040 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:31.754015    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45041402_dc9b_404d_a815_99e68f9f1e21.slice/crio-c68c1e3d3b4f44975947de9fab0e8dfc4ff443d54b7abd05d6d394d8908fab50 WatchSource:0}: Error finding container c68c1e3d3b4f44975947de9fab0e8dfc4ff443d54b7abd05d6d394d8908fab50: Status 404 returned error can't find the container with id c68c1e3d3b4f44975947de9fab0e8dfc4ff443d54b7abd05d6d394d8908fab50
Jul 15 21:03:31.768300 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.768267    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99"
Jul 15 21:03:31.905202 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:31.905171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-7gk99"]
Jul 15 21:03:31.908488 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:31.908459    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254e79cf_8922_4bf5_a354_5f08af660818.slice/crio-d117bb36036ff3c6ac860fabf5b596514bab5da7b9622a149535daabaf1737c2 WatchSource:0}: Error finding container d117bb36036ff3c6ac860fabf5b596514bab5da7b9622a149535daabaf1737c2: Status 404 returned error can't find the container with id d117bb36036ff3c6ac860fabf5b596514bab5da7b9622a149535daabaf1737c2
Jul 15 21:03:32.597741 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:32.597701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99" event={"ID":"254e79cf-8922-4bf5-a354-5f08af660818","Type":"ContainerStarted","Data":"d117bb36036ff3c6ac860fabf5b596514bab5da7b9622a149535daabaf1737c2"}
Jul 15 21:03:32.599728 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:32.599691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4z7rk" event={"ID":"45041402-dc9b-404d-a815-99e68f9f1e21","Type":"ContainerStarted","Data":"c68c1e3d3b4f44975947de9fab0e8dfc4ff443d54b7abd05d6d394d8908fab50"}
Jul 15 21:03:33.605791 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:33.605702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99" event={"ID":"254e79cf-8922-4bf5-a354-5f08af660818","Type":"ContainerStarted","Data":"3bb824be08ea2c27d62a650114f5c48bf3e669bfb13bf15c762436f5722b4b57"}
Jul 15 21:03:33.606266 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:33.605803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99" event={"ID":"254e79cf-8922-4bf5-a354-5f08af660818","Type":"ContainerStarted","Data":"023fad107f26c2e30c804b46b0cad51cde484e3e6481513b0032385ec25efeeb"}
Jul 15 21:03:33.606437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:33.606404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99" event={"ID":"254e79cf-8922-4bf5-a354-5f08af660818","Type":"ContainerStarted","Data":"99c9d3340249a14b34ec6d165496aab939ec3a769fe379f5e72c21905a73d389"}
Jul 15 21:03:33.607803 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:33.607778    2580 generic.go:356] "Generic (PLEG): container finished" podID="45041402-dc9b-404d-a815-99e68f9f1e21" containerID="bbcdc7396b1e060f6754c45f4738bd66db925cfa2c19ebc9130b47946e7ec32b" exitCode=0
Jul 15 21:03:33.607905 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:33.607857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4z7rk" event={"ID":"45041402-dc9b-404d-a815-99e68f9f1e21","Type":"ContainerDied","Data":"bbcdc7396b1e060f6754c45f4738bd66db925cfa2c19ebc9130b47946e7ec32b"}
Jul 15 21:03:33.629210 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:33.629039    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-7gk99" podStartSLOduration=1.159616337 podStartE2EDuration="2.629015232s" totalImagesPullingTime="1.469398895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:33.62738401 +0000 UTC m=+82.984302606" watchObservedRunningTime="2026-07-15 21:03:33.629015232 +0000 UTC m=+82.985933829"
Jul 15 21:03:34.511145 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.511109    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"]
Jul 15 21:03:34.514992 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.514948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.517630 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.517606    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 15 21:03:34.518132 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.518110    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-b3olvt9r5a0p4\""
Jul 15 21:03:34.518252 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.518134    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 15 21:03:34.518364 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.518346    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 15 21:03:34.518445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.518346    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 15 21:03:34.518582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.518566    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 15 21:03:34.518648 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.518635    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-s5wd2\""
Jul 15 21:03:34.532340 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.532309    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"]
Jul 15 21:03:34.614260 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.614225    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4z7rk" event={"ID":"45041402-dc9b-404d-a815-99e68f9f1e21","Type":"ContainerStarted","Data":"1c21bfee6b6c8caab97a27681b5f02300e6ead8a3b6c6e0208e35bb23f0f11c4"}
Jul 15 21:03:34.614639 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.614314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4z7rk" event={"ID":"45041402-dc9b-404d-a815-99e68f9f1e21","Type":"ContainerStarted","Data":"69c4a257b05d590eab7d6cd41489b10c1691a9665588761456e137d01e0a5ece"}
Jul 15 21:03:34.643949 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.643879    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4z7rk" podStartSLOduration=2.601102007 podStartE2EDuration="3.643861019s" totalImagesPullingTime="969.259758ms" totalInitContainerRuntime="73.499254ms" isStatefulPod=true podCreationTimestamp="2026-07-15 21:03:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:34.642374792 +0000 UTC m=+83.999293389" watchObservedRunningTime="2026-07-15 21:03:34.643861019 +0000 UTC m=+84.000779614"
Jul 15 21:03:34.678909 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.678875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.678909 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.678910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.679106 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.678932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-grpc-tls\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.679106 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.678983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74bce824-176c-460a-bcb5-362067d306f4-metrics-client-ca\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.679106 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.679007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75qhr\" (UniqueName: \"kubernetes.io/projected/74bce824-176c-460a-bcb5-362067d306f4-kube-api-access-75qhr\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.679106 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.679026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-tls\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.679106 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.679044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.679249 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.679107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.780550 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.780519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74bce824-176c-460a-bcb5-362067d306f4-metrics-client-ca\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.780704 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.780556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75qhr\" (UniqueName: \"kubernetes.io/projected/74bce824-176c-460a-bcb5-362067d306f4-kube-api-access-75qhr\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.780758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.780689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-tls\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.780758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.780738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.780878 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.780778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.781483 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.780989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.781483 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.781037    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.781483 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.781088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-grpc-tls\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.781483 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.781339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74bce824-176c-460a-bcb5-362067d306f4-metrics-client-ca\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.783590 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.783565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.783800 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.783781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-tls\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.783864 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.783847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.783915 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.783886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.784123 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.784106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-grpc-tls\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.784233 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.784214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74bce824-176c-460a-bcb5-362067d306f4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.789105 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.789087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75qhr\" (UniqueName: \"kubernetes.io/projected/74bce824-176c-460a-bcb5-362067d306f4-kube-api-access-75qhr\") pod \"thanos-querier-786b5d98d4-hrzlc\" (UID: \"74bce824-176c-460a-bcb5-362067d306f4\") " pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.824116 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.824088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:34.967423 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:34.967395    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"]
Jul 15 21:03:34.969267 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:34.969242    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74bce824_176c_460a_bcb5_362067d306f4.slice/crio-295dcb690ec5b396ad119958e7aa979768ebfb094238583ba1bd69e4ffd3e28a WatchSource:0}: Error finding container 295dcb690ec5b396ad119958e7aa979768ebfb094238583ba1bd69e4ffd3e28a: Status 404 returned error can't find the container with id 295dcb690ec5b396ad119958e7aa979768ebfb094238583ba1bd69e4ffd3e28a
Jul 15 21:03:35.620606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.620566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc" event={"ID":"74bce824-176c-460a-bcb5-362067d306f4","Type":"ContainerStarted","Data":"295dcb690ec5b396ad119958e7aa979768ebfb094238583ba1bd69e4ffd3e28a"}
Jul 15 21:03:35.843978 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.843928    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6f78695788-j9vmj"]
Jul 15 21:03:35.847286 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.847263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:35.850906 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.850882    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 15 21:03:35.851194 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.851174    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 15 21:03:35.851680 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.851633    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 21:03:35.851803 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.851688    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-b7xgq\""
Jul 15 21:03:35.852146 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.852124    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fgh2k4024n032\""
Jul 15 21:03:35.852146 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.852132    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 15 21:03:35.859769 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.859745    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f78695788-j9vmj"]
Jul 15 21:03:35.992363 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.992273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adff73c0-7575-47dd-a9cf-6323394c14fa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:35.992363 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.992327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmnsw\" (UniqueName: \"kubernetes.io/projected/adff73c0-7575-47dd-a9cf-6323394c14fa-kube-api-access-jmnsw\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:35.992555 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.992400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/adff73c0-7575-47dd-a9cf-6323394c14fa-metrics-server-audit-profiles\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:35.992555 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.992456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/adff73c0-7575-47dd-a9cf-6323394c14fa-secret-metrics-server-client-certs\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:35.992555 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.992494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adff73c0-7575-47dd-a9cf-6323394c14fa-client-ca-bundle\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:35.992555 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.992539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/adff73c0-7575-47dd-a9cf-6323394c14fa-audit-log\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:35.992736 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:35.992601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/adff73c0-7575-47dd-a9cf-6323394c14fa-secret-metrics-server-tls\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.094068 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.094035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/adff73c0-7575-47dd-a9cf-6323394c14fa-metrics-server-audit-profiles\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.094068 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.094070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/adff73c0-7575-47dd-a9cf-6323394c14fa-secret-metrics-server-client-certs\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.094289 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.094088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adff73c0-7575-47dd-a9cf-6323394c14fa-client-ca-bundle\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.094289 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.094117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/adff73c0-7575-47dd-a9cf-6323394c14fa-audit-log\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.094289 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.094145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/adff73c0-7575-47dd-a9cf-6323394c14fa-secret-metrics-server-tls\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.094289 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.094201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adff73c0-7575-47dd-a9cf-6323394c14fa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.094289 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.094229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmnsw\" (UniqueName: \"kubernetes.io/projected/adff73c0-7575-47dd-a9cf-6323394c14fa-kube-api-access-jmnsw\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.094650 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.094624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/adff73c0-7575-47dd-a9cf-6323394c14fa-audit-log\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.094990 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.094941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adff73c0-7575-47dd-a9cf-6323394c14fa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.095251 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.095234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/adff73c0-7575-47dd-a9cf-6323394c14fa-metrics-server-audit-profiles\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.096562 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.096535    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/adff73c0-7575-47dd-a9cf-6323394c14fa-secret-metrics-server-client-certs\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.096677 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.096634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adff73c0-7575-47dd-a9cf-6323394c14fa-client-ca-bundle\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.096763 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.096743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/adff73c0-7575-47dd-a9cf-6323394c14fa-secret-metrics-server-tls\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.104367 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.104341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmnsw\" (UniqueName: \"kubernetes.io/projected/adff73c0-7575-47dd-a9cf-6323394c14fa-kube-api-access-jmnsw\") pod \"metrics-server-6f78695788-j9vmj\" (UID: \"adff73c0-7575-47dd-a9cf-6323394c14fa\") " pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.159820 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.159691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:36.301031 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:36.301007    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f78695788-j9vmj"]
Jul 15 21:03:36.741848 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:36.741810    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadff73c0_7575_47dd_a9cf_6323394c14fa.slice/crio-5717b071d89af936e2e2abf85f4c0b1e1860c0d1cd9ce722cf041bd907ef3c37 WatchSource:0}: Error finding container 5717b071d89af936e2e2abf85f4c0b1e1860c0d1cd9ce722cf041bd907ef3c37: Status 404 returned error can't find the container with id 5717b071d89af936e2e2abf85f4c0b1e1860c0d1cd9ce722cf041bd907ef3c37
Jul 15 21:03:37.631410 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:37.631324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f78695788-j9vmj" event={"ID":"adff73c0-7575-47dd-a9cf-6323394c14fa","Type":"ContainerStarted","Data":"5717b071d89af936e2e2abf85f4c0b1e1860c0d1cd9ce722cf041bd907ef3c37"}
Jul 15 21:03:37.633872 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:37.633839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc" event={"ID":"74bce824-176c-460a-bcb5-362067d306f4","Type":"ContainerStarted","Data":"285d1652e5c59214825d03ab7cb2299fbf3951cefa5f968756c6848e2321c787"}
Jul 15 21:03:37.634024 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:37.633903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc" event={"ID":"74bce824-176c-460a-bcb5-362067d306f4","Type":"ContainerStarted","Data":"5e3f97bc691c21d66e1875c0c67b758c2833acab48adc9b1b2ea891c992c3a09"}
Jul 15 21:03:37.634024 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:37.633932    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc" event={"ID":"74bce824-176c-460a-bcb5-362067d306f4","Type":"ContainerStarted","Data":"8908bfb7e6d8b9f3fdb320835f6d5c33f1775e10d0edb8eba98b6be3fa8ebccd"}
Jul 15 21:03:38.639203 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:38.639162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f78695788-j9vmj" event={"ID":"adff73c0-7575-47dd-a9cf-6323394c14fa","Type":"ContainerStarted","Data":"c503f2bbc6c05f3208e13ab3bfd7eabbdae06cb7193c918b6501a17cfba361f1"}
Jul 15 21:03:38.642321 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:38.642295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc" event={"ID":"74bce824-176c-460a-bcb5-362067d306f4","Type":"ContainerStarted","Data":"ae0f9e82ccae8047a7a96f0c0a9821779b8dc4b7559a38887e263ead10c6541b"}
Jul 15 21:03:38.642423 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:38.642352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc" event={"ID":"74bce824-176c-460a-bcb5-362067d306f4","Type":"ContainerStarted","Data":"2250edca7ae03b19d57e28d5703e75e09ccf2e424caba482fbb530f60e57491a"}
Jul 15 21:03:38.642423 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:38.642378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc" event={"ID":"74bce824-176c-460a-bcb5-362067d306f4","Type":"ContainerStarted","Data":"425854d103a610144db5f386e96852e7b825e8d2e7b9d2fda294721951259f88"}
Jul 15 21:03:38.642423 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:38.642406    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:38.664793 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:38.664737    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6f78695788-j9vmj" podStartSLOduration=2.242295767 podStartE2EDuration="3.664721185s" totalImagesPullingTime="1.422425418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:38.662989446 +0000 UTC m=+88.019908041" watchObservedRunningTime="2026-07-15 21:03:38.664721185 +0000 UTC m=+88.021639780"
Jul 15 21:03:38.700531 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:38.700470    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc" podStartSLOduration=1.6921948489999998 podStartE2EDuration="4.700447041s" totalImagesPullingTime="3.008252192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:38.698028194 +0000 UTC m=+88.054946791" watchObservedRunningTime="2026-07-15 21:03:38.700447041 +0000 UTC m=+88.057365637"
Jul 15 21:03:39.234010 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.233979    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d75cc5f7d-8qnjh"]
Jul 15 21:03:39.236362 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.236345    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.239745 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.239721    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 21:03:39.239861 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.239756    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 21:03:39.240036 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.240012    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 21:03:39.240036 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.240026    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 21:03:39.240166 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.240034    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 21:03:39.240166 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.240112    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 21:03:39.240166 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.240037    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 21:03:39.240291 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.240169    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6gzxm\""
Jul 15 21:03:39.254910 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.254890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d75cc5f7d-8qnjh"]
Jul 15 21:03:39.428422 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.428383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-oauth-config\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.428422 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.428432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-console-config\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.428625 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.428463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-serving-cert\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.428625 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.428502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-service-ca\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.428625 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.428538    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m46c\" (UniqueName: \"kubernetes.io/projected/70470d2c-9f3c-4859-b753-04cee7262c78-kube-api-access-8m46c\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.428625 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.428559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-oauth-serving-cert\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.530067 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.530032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-oauth-config\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.530295 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.530096    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-console-config\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.530295 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.530138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-serving-cert\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.530295 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.530161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-service-ca\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.530295 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.530182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m46c\" (UniqueName: \"kubernetes.io/projected/70470d2c-9f3c-4859-b753-04cee7262c78-kube-api-access-8m46c\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.530295 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.530206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-oauth-serving-cert\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.531042 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.531016    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-oauth-serving-cert\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.531222 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.531199    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-service-ca\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.531276 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.531259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-console-config\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.533042 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.533023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-oauth-config\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.533130 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.533094    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-serving-cert\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.543889 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.543866    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85bc5987-9hdxw"
Jul 15 21:03:39.545842 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.545820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m46c\" (UniqueName: \"kubernetes.io/projected/70470d2c-9f3c-4859-b753-04cee7262c78-kube-api-access-8m46c\") pod \"console-7d75cc5f7d-8qnjh\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") " pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.845653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.845565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:39.980159 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:39.980133    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d75cc5f7d-8qnjh"]
Jul 15 21:03:39.981839 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:39.981810    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70470d2c_9f3c_4859_b753_04cee7262c78.slice/crio-3d0ed542e537e1e1ff42e70cc5443b9620c1c48e48c44d9e86b99f14feb91f4a WatchSource:0}: Error finding container 3d0ed542e537e1e1ff42e70cc5443b9620c1c48e48c44d9e86b99f14feb91f4a: Status 404 returned error can't find the container with id 3d0ed542e537e1e1ff42e70cc5443b9620c1c48e48c44d9e86b99f14feb91f4a
Jul 15 21:03:40.651827 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:40.651791    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d75cc5f7d-8qnjh" event={"ID":"70470d2c-9f3c-4859-b753-04cee7262c78","Type":"ContainerStarted","Data":"3d0ed542e537e1e1ff42e70cc5443b9620c1c48e48c44d9e86b99f14feb91f4a"}
Jul 15 21:03:41.797998 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:41.797930    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q" podUID="72565261-8a62-4c4e-9290-ce35e8e1abad" containerName="registry" containerID="cri-o://3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab" gracePeriod=30
Jul 15 21:03:42.058866 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.058805    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:03:42.252946 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.252913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-bound-sa-token\") pod \"72565261-8a62-4c4e-9290-ce35e8e1abad\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") "
Jul 15 21:03:42.253119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.252971    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-image-registry-private-configuration\") pod \"72565261-8a62-4c4e-9290-ce35e8e1abad\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") "
Jul 15 21:03:42.253119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.252998    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-trusted-ca\") pod \"72565261-8a62-4c4e-9290-ce35e8e1abad\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") "
Jul 15 21:03:42.253119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.253030    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") pod \"72565261-8a62-4c4e-9290-ce35e8e1abad\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") "
Jul 15 21:03:42.253119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.253062    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72565261-8a62-4c4e-9290-ce35e8e1abad-ca-trust-extracted\") pod \"72565261-8a62-4c4e-9290-ce35e8e1abad\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") "
Jul 15 21:03:42.253119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.253084    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-installation-pull-secrets\") pod \"72565261-8a62-4c4e-9290-ce35e8e1abad\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") "
Jul 15 21:03:42.253361 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.253125    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkqfz\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-kube-api-access-qkqfz\") pod \"72565261-8a62-4c4e-9290-ce35e8e1abad\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") "
Jul 15 21:03:42.253361 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.253150    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-certificates\") pod \"72565261-8a62-4c4e-9290-ce35e8e1abad\" (UID: \"72565261-8a62-4c4e-9290-ce35e8e1abad\") "
Jul 15 21:03:42.253730 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.253520    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "72565261-8a62-4c4e-9290-ce35e8e1abad" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:03:42.253915 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.253890    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "72565261-8a62-4c4e-9290-ce35e8e1abad" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:03:42.256401 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.256135    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "72565261-8a62-4c4e-9290-ce35e8e1abad" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:03:42.256401 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.256201    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "72565261-8a62-4c4e-9290-ce35e8e1abad" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:03:42.256401 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.256347    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "72565261-8a62-4c4e-9290-ce35e8e1abad" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:03:42.256811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.256786    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-kube-api-access-qkqfz" (OuterVolumeSpecName: "kube-api-access-qkqfz") pod "72565261-8a62-4c4e-9290-ce35e8e1abad" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad"). InnerVolumeSpecName "kube-api-access-qkqfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:03:42.256880 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.256850    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "72565261-8a62-4c4e-9290-ce35e8e1abad" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:03:42.262422 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.262398    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72565261-8a62-4c4e-9290-ce35e8e1abad-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "72565261-8a62-4c4e-9290-ce35e8e1abad" (UID: "72565261-8a62-4c4e-9290-ce35e8e1abad"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:03:42.354713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.354612    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-bound-sa-token\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:03:42.354713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.354654    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-image-registry-private-configuration\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:03:42.354713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.354670    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-trusted-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:03:42.354713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.354682    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:03:42.354713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.354690    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72565261-8a62-4c4e-9290-ce35e8e1abad-ca-trust-extracted\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:03:42.354713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.354709    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72565261-8a62-4c4e-9290-ce35e8e1abad-installation-pull-secrets\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:03:42.354713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.354719    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkqfz\" (UniqueName: \"kubernetes.io/projected/72565261-8a62-4c4e-9290-ce35e8e1abad-kube-api-access-qkqfz\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:03:42.355119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.354727    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72565261-8a62-4c4e-9290-ce35e8e1abad-registry-certificates\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:03:42.661724 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.661634    2580 generic.go:356] "Generic (PLEG): container finished" podID="72565261-8a62-4c4e-9290-ce35e8e1abad" containerID="3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab" exitCode=0
Jul 15 21:03:42.661724 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.661690    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q"
Jul 15 21:03:42.661974 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.661727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q" event={"ID":"72565261-8a62-4c4e-9290-ce35e8e1abad","Type":"ContainerDied","Data":"3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab"}
Jul 15 21:03:42.661974 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.661792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8f8c6575-98f4q" event={"ID":"72565261-8a62-4c4e-9290-ce35e8e1abad","Type":"ContainerDied","Data":"5ea95bd1ba6d547cae595b3a44b17960071f1a289a59c881a9f1f69c5dea55c3"}
Jul 15 21:03:42.661974 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.661820    2580 scope.go:117] "RemoveContainer" containerID="3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab"
Jul 15 21:03:42.692348 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.692316    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d8f8c6575-98f4q"]
Jul 15 21:03:42.700776 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:42.700747    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d8f8c6575-98f4q"]
Jul 15 21:03:43.135089 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:43.135055    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72565261-8a62-4c4e-9290-ce35e8e1abad" path="/var/lib/kubelet/pods/72565261-8a62-4c4e-9290-ce35e8e1abad/volumes"
Jul 15 21:03:43.293157 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:43.292930    2580 scope.go:117] "RemoveContainer" containerID="3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab"
Jul 15 21:03:43.293380 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:03:43.293356    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab\": container with ID starting with 3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab not found: ID does not exist" containerID="3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab"
Jul 15 21:03:43.293435 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:43.293392    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab"} err="failed to get container status \"3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab\": rpc error: code = NotFound desc = could not find container \"3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab\": container with ID starting with 3c0d9a90ea59d0bbf5f94ebc9d662704148abf822b4955d891794b743563ebab not found: ID does not exist"
Jul 15 21:03:43.667242 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:43.667144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d75cc5f7d-8qnjh" event={"ID":"70470d2c-9f3c-4859-b753-04cee7262c78","Type":"ContainerStarted","Data":"e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e"}
Jul 15 21:03:43.688155 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:43.688091    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d75cc5f7d-8qnjh" podStartSLOduration=1.329760072 podStartE2EDuration="4.688071239s" totalImagesPullingTime="3.358311167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:43.687024778 +0000 UTC m=+93.043943374" watchObservedRunningTime="2026-07-15 21:03:43.688071239 +0000 UTC m=+93.044989835"
Jul 15 21:03:44.653398 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:44.653368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-786b5d98d4-hrzlc"
Jul 15 21:03:47.518489 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:47.518459    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kmgwk"
Jul 15 21:03:48.386742 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.386696    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68794bd79c-88pvn"]
Jul 15 21:03:48.387062 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.387049    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72565261-8a62-4c4e-9290-ce35e8e1abad" containerName="registry"
Jul 15 21:03:48.387104 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.387064    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="72565261-8a62-4c4e-9290-ce35e8e1abad" containerName="registry"
Jul 15 21:03:48.387133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.387116    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="72565261-8a62-4c4e-9290-ce35e8e1abad" containerName="registry"
Jul 15 21:03:48.651565 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.651490    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68794bd79c-88pvn"]
Jul 15 21:03:48.651991 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.651596    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.659716 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.659686    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 21:03:48.702841 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.702810    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-service-ca\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.702841 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.702844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-console-config\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.703100 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.702861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-oauth-config\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.703100 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.702884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-oauth-serving-cert\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.703100 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.702939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-trusted-ca-bundle\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.703100 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.702993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-serving-cert\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.703100 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.703023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcvdr\" (UniqueName: \"kubernetes.io/projected/040ebfaf-d53e-477a-81bf-9068decec17e-kube-api-access-pcvdr\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.804238 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.804200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcvdr\" (UniqueName: \"kubernetes.io/projected/040ebfaf-d53e-477a-81bf-9068decec17e-kube-api-access-pcvdr\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.804439 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.804264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-service-ca\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.804439 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.804292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-console-config\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.804439 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.804314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-oauth-config\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.804439 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.804340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-oauth-serving-cert\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.804439 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.804378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-trusted-ca-bundle\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.804439 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.804404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-serving-cert\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.805125 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.805096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-service-ca\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.805250 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.805105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-console-config\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.805250 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.805214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-oauth-serving-cert\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.805324 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.805300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-trusted-ca-bundle\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.807446 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.807408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-serving-cert\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.807539 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.807522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-oauth-config\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.813915 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.813890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcvdr\" (UniqueName: \"kubernetes.io/projected/040ebfaf-d53e-477a-81bf-9068decec17e-kube-api-access-pcvdr\") pod \"console-68794bd79c-88pvn\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") " pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:48.961345 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:48.961246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:49.142696 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:49.142666    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68794bd79c-88pvn"]
Jul 15 21:03:49.146346 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:03:49.146317    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040ebfaf_d53e_477a_81bf_9068decec17e.slice/crio-9d35ac3497f771c6b4468d98ced9b700cacda953f56ed182cf6c1632fc23b1bf WatchSource:0}: Error finding container 9d35ac3497f771c6b4468d98ced9b700cacda953f56ed182cf6c1632fc23b1bf: Status 404 returned error can't find the container with id 9d35ac3497f771c6b4468d98ced9b700cacda953f56ed182cf6c1632fc23b1bf
Jul 15 21:03:49.695193 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:49.695158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68794bd79c-88pvn" event={"ID":"040ebfaf-d53e-477a-81bf-9068decec17e","Type":"ContainerStarted","Data":"e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129"}
Jul 15 21:03:49.695641 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:49.695214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68794bd79c-88pvn" event={"ID":"040ebfaf-d53e-477a-81bf-9068decec17e","Type":"ContainerStarted","Data":"9d35ac3497f771c6b4468d98ced9b700cacda953f56ed182cf6c1632fc23b1bf"}
Jul 15 21:03:49.722659 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:49.722603    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68794bd79c-88pvn" podStartSLOduration=1.722585941 podStartE2EDuration="1.722585941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:49.721454591 +0000 UTC m=+99.078373187" watchObservedRunningTime="2026-07-15 21:03:49.722585941 +0000 UTC m=+99.079504538"
Jul 15 21:03:49.846404 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:49.846368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:49.846734 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:49.846702    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:49.851330 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:49.851306    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:50.703167 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:50.703136    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:03:56.160568 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:56.160520    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:56.161096 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:56.160616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:03:58.962508 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:58.962466    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:58.962937 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:58.962532    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:58.968088 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:58.968063    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:59.737996 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:59.737950    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:03:59.807873 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:03:59.807842    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d75cc5f7d-8qnjh"]
Jul 15 21:04:11.781224 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:11.781191    2580 generic.go:356] "Generic (PLEG): container finished" podID="1d90ed07-ed6a-458d-956a-1da3260948d2" containerID="c489e21e126debcb6b2104f941858708a9c095a89512f1afab597d68f7e94373" exitCode=0
Jul 15 21:04:11.781611 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:11.781229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l" event={"ID":"1d90ed07-ed6a-458d-956a-1da3260948d2","Type":"ContainerDied","Data":"c489e21e126debcb6b2104f941858708a9c095a89512f1afab597d68f7e94373"}
Jul 15 21:04:11.781611 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:11.781604    2580 scope.go:117] "RemoveContainer" containerID="c489e21e126debcb6b2104f941858708a9c095a89512f1afab597d68f7e94373"
Jul 15 21:04:12.786608 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:12.786574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-dx82l" event={"ID":"1d90ed07-ed6a-458d-956a-1da3260948d2","Type":"ContainerStarted","Data":"3ade8a52467d97e64c1cd32c16e3b424ef4ca1d84abafc164c139a285677eaf7"}
Jul 15 21:04:16.165836 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:16.165796    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:04:16.169575 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:16.169549    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6f78695788-j9vmj"
Jul 15 21:04:24.829560 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:24.829480    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d75cc5f7d-8qnjh" podUID="70470d2c-9f3c-4859-b753-04cee7262c78" containerName="console" containerID="cri-o://e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e" gracePeriod=15
Jul 15 21:04:25.105405 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.105371    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:04:25.133020 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.131360    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-service-ca\") pod \"70470d2c-9f3c-4859-b753-04cee7262c78\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") "
Jul 15 21:04:25.133020 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.131439    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-oauth-serving-cert\") pod \"70470d2c-9f3c-4859-b753-04cee7262c78\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") "
Jul 15 21:04:25.133020 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.131469    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-console-config\") pod \"70470d2c-9f3c-4859-b753-04cee7262c78\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") "
Jul 15 21:04:25.133020 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.131509    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-serving-cert\") pod \"70470d2c-9f3c-4859-b753-04cee7262c78\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") "
Jul 15 21:04:25.133020 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.131539    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m46c\" (UniqueName: \"kubernetes.io/projected/70470d2c-9f3c-4859-b753-04cee7262c78-kube-api-access-8m46c\") pod \"70470d2c-9f3c-4859-b753-04cee7262c78\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") "
Jul 15 21:04:25.133020 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.131585    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-oauth-config\") pod \"70470d2c-9f3c-4859-b753-04cee7262c78\" (UID: \"70470d2c-9f3c-4859-b753-04cee7262c78\") "
Jul 15 21:04:25.133464 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.133290    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-console-config" (OuterVolumeSpecName: "console-config") pod "70470d2c-9f3c-4859-b753-04cee7262c78" (UID: "70470d2c-9f3c-4859-b753-04cee7262c78"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:04:25.134300 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.133542    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-service-ca" (OuterVolumeSpecName: "service-ca") pod "70470d2c-9f3c-4859-b753-04cee7262c78" (UID: "70470d2c-9f3c-4859-b753-04cee7262c78"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:04:25.134300 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.133779    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "70470d2c-9f3c-4859-b753-04cee7262c78" (UID: "70470d2c-9f3c-4859-b753-04cee7262c78"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:04:25.144873 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.144752    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70470d2c-9f3c-4859-b753-04cee7262c78-kube-api-access-8m46c" (OuterVolumeSpecName: "kube-api-access-8m46c") pod "70470d2c-9f3c-4859-b753-04cee7262c78" (UID: "70470d2c-9f3c-4859-b753-04cee7262c78"). InnerVolumeSpecName "kube-api-access-8m46c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:04:25.144873 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.144854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "70470d2c-9f3c-4859-b753-04cee7262c78" (UID: "70470d2c-9f3c-4859-b753-04cee7262c78"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:04:25.145309 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.145284    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "70470d2c-9f3c-4859-b753-04cee7262c78" (UID: "70470d2c-9f3c-4859-b753-04cee7262c78"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:04:25.232324 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.232289    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:04:25.232324 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.232321    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-oauth-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:04:25.232324 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.232330    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70470d2c-9f3c-4859-b753-04cee7262c78-console-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:04:25.232626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.232339    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:04:25.232626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.232348    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m46c\" (UniqueName: \"kubernetes.io/projected/70470d2c-9f3c-4859-b753-04cee7262c78-kube-api-access-8m46c\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:04:25.232626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.232357    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70470d2c-9f3c-4859-b753-04cee7262c78-console-oauth-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:04:25.840515 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.840470    2580 generic.go:356] "Generic (PLEG): container finished" podID="70470d2c-9f3c-4859-b753-04cee7262c78" containerID="e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e" exitCode=0
Jul 15 21:04:25.840515 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.840516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d75cc5f7d-8qnjh" event={"ID":"70470d2c-9f3c-4859-b753-04cee7262c78","Type":"ContainerDied","Data":"e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e"}
Jul 15 21:04:25.841116 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.840532    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d75cc5f7d-8qnjh"
Jul 15 21:04:25.841116 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.840583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d75cc5f7d-8qnjh" event={"ID":"70470d2c-9f3c-4859-b753-04cee7262c78","Type":"ContainerDied","Data":"3d0ed542e537e1e1ff42e70cc5443b9620c1c48e48c44d9e86b99f14feb91f4a"}
Jul 15 21:04:25.841116 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.840603    2580 scope.go:117] "RemoveContainer" containerID="e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e"
Jul 15 21:04:25.849255 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.849223    2580 scope.go:117] "RemoveContainer" containerID="e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e"
Jul 15 21:04:25.849537 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:04:25.849518    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e\": container with ID starting with e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e not found: ID does not exist" containerID="e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e"
Jul 15 21:04:25.849601 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.849549    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e"} err="failed to get container status \"e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e\": rpc error: code = NotFound desc = could not find container \"e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e\": container with ID starting with e2dfc519142fedf2ce1d8a214318b9f7896bb228f5323fa487d220820ea20f5e not found: ID does not exist"
Jul 15 21:04:25.857684 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.857634    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d75cc5f7d-8qnjh"]
Jul 15 21:04:25.861828 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:25.861806    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d75cc5f7d-8qnjh"]
Jul 15 21:04:27.133745 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:04:27.133708    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70470d2c-9f3c-4859-b753-04cee7262c78" path="/var/lib/kubelet/pods/70470d2c-9f3c-4859-b753-04cee7262c78/volumes"
Jul 15 21:05:00.627268 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.627231    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54488fd664-gctvz"]
Jul 15 21:05:00.628035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.628011    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70470d2c-9f3c-4859-b753-04cee7262c78" containerName="console"
Jul 15 21:05:00.628035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.628035    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="70470d2c-9f3c-4859-b753-04cee7262c78" containerName="console"
Jul 15 21:05:00.628284 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.628115    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="70470d2c-9f3c-4859-b753-04cee7262c78" containerName="console"
Jul 15 21:05:00.632439 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.632405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.641303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.641278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54488fd664-gctvz"]
Jul 15 21:05:00.737715 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.737674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-oauth-config\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.737715 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.737718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-oauth-serving-cert\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.737950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.737748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-serving-cert\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.737950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.737795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-trusted-ca-bundle\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.737950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.737854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7mpd\" (UniqueName: \"kubernetes.io/projected/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-kube-api-access-q7mpd\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.737950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.737887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-config\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.737950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.737908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-service-ca\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.838612 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.838565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-oauth-serving-cert\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.838811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.838629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-serving-cert\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.838811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.838646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-trusted-ca-bundle\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.838811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.838671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7mpd\" (UniqueName: \"kubernetes.io/projected/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-kube-api-access-q7mpd\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.838811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.838693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-config\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.838811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.838715    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-service-ca\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.838811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.838749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-oauth-config\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.839929 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.839862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-service-ca\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.839929 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.839869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-oauth-serving-cert\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.840135 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.840067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-config\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.840189 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.840144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-trusted-ca-bundle\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.842132 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.842104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-oauth-config\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.845798 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.845771    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-serving-cert\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.847870 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.847847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7mpd\" (UniqueName: \"kubernetes.io/projected/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-kube-api-access-q7mpd\") pod \"console-54488fd664-gctvz\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") " pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:00.943114 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:00.943018    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:01.076230 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:01.076084    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54488fd664-gctvz"]
Jul 15 21:05:01.079368 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:05:01.079336    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod499d76ff_1f25_42c9_afcc_9ef76ea0f89a.slice/crio-5dd6b359e830c54464e4ed9a8336fd3091bcb6aea3373008a0cdc61558cb8d65 WatchSource:0}: Error finding container 5dd6b359e830c54464e4ed9a8336fd3091bcb6aea3373008a0cdc61558cb8d65: Status 404 returned error can't find the container with id 5dd6b359e830c54464e4ed9a8336fd3091bcb6aea3373008a0cdc61558cb8d65
Jul 15 21:05:01.981486 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:01.981411    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54488fd664-gctvz" event={"ID":"499d76ff-1f25-42c9-afcc-9ef76ea0f89a","Type":"ContainerStarted","Data":"217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98"}
Jul 15 21:05:01.981486 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:01.981460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54488fd664-gctvz" event={"ID":"499d76ff-1f25-42c9-afcc-9ef76ea0f89a","Type":"ContainerStarted","Data":"5dd6b359e830c54464e4ed9a8336fd3091bcb6aea3373008a0cdc61558cb8d65"}
Jul 15 21:05:02.002151 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:02.001979    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54488fd664-gctvz" podStartSLOduration=2.001936895 podStartE2EDuration="2.001936895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:05:02.001670825 +0000 UTC m=+171.358589423" watchObservedRunningTime="2026-07-15 21:05:02.001936895 +0000 UTC m=+171.358855493"
Jul 15 21:05:10.943159 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:10.943108    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:10.943755 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:10.943182    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:10.949076 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:10.949046    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:11.019106 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:11.019076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:05:11.077285 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:11.077254    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68794bd79c-88pvn"]
Jul 15 21:05:36.103756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.103682    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68794bd79c-88pvn" podUID="040ebfaf-d53e-477a-81bf-9068decec17e" containerName="console" containerID="cri-o://e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129" gracePeriod=15
Jul 15 21:05:36.354751 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.354685    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:05:36.427065 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427024    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcvdr\" (UniqueName: \"kubernetes.io/projected/040ebfaf-d53e-477a-81bf-9068decec17e-kube-api-access-pcvdr\") pod \"040ebfaf-d53e-477a-81bf-9068decec17e\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") "
Jul 15 21:05:36.427239 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427102    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-serving-cert\") pod \"040ebfaf-d53e-477a-81bf-9068decec17e\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") "
Jul 15 21:05:36.427239 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427155    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-oauth-config\") pod \"040ebfaf-d53e-477a-81bf-9068decec17e\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") "
Jul 15 21:05:36.427239 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-oauth-serving-cert\") pod \"040ebfaf-d53e-477a-81bf-9068decec17e\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") "
Jul 15 21:05:36.427239 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427209    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-trusted-ca-bundle\") pod \"040ebfaf-d53e-477a-81bf-9068decec17e\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") "
Jul 15 21:05:36.427445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427247    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-console-config\") pod \"040ebfaf-d53e-477a-81bf-9068decec17e\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") "
Jul 15 21:05:36.427445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427286    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-service-ca\") pod \"040ebfaf-d53e-477a-81bf-9068decec17e\" (UID: \"040ebfaf-d53e-477a-81bf-9068decec17e\") "
Jul 15 21:05:36.427661 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427619    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "040ebfaf-d53e-477a-81bf-9068decec17e" (UID: "040ebfaf-d53e-477a-81bf-9068decec17e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:05:36.427661 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427569    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "040ebfaf-d53e-477a-81bf-9068decec17e" (UID: "040ebfaf-d53e-477a-81bf-9068decec17e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:05:36.427855 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427803    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-service-ca" (OuterVolumeSpecName: "service-ca") pod "040ebfaf-d53e-477a-81bf-9068decec17e" (UID: "040ebfaf-d53e-477a-81bf-9068decec17e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:05:36.427899 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.427843    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-console-config" (OuterVolumeSpecName: "console-config") pod "040ebfaf-d53e-477a-81bf-9068decec17e" (UID: "040ebfaf-d53e-477a-81bf-9068decec17e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:05:36.429309 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.429289    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "040ebfaf-d53e-477a-81bf-9068decec17e" (UID: "040ebfaf-d53e-477a-81bf-9068decec17e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:05:36.429607 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.429589    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "040ebfaf-d53e-477a-81bf-9068decec17e" (UID: "040ebfaf-d53e-477a-81bf-9068decec17e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:05:36.429607 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.429594    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040ebfaf-d53e-477a-81bf-9068decec17e-kube-api-access-pcvdr" (OuterVolumeSpecName: "kube-api-access-pcvdr") pod "040ebfaf-d53e-477a-81bf-9068decec17e" (UID: "040ebfaf-d53e-477a-81bf-9068decec17e"). InnerVolumeSpecName "kube-api-access-pcvdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:05:36.528484 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.528449    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:05:36.528484 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.528479    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcvdr\" (UniqueName: \"kubernetes.io/projected/040ebfaf-d53e-477a-81bf-9068decec17e-kube-api-access-pcvdr\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:05:36.528484 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.528492    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:05:36.528722 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.528502    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/040ebfaf-d53e-477a-81bf-9068decec17e-console-oauth-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:05:36.528722 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.528511    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-oauth-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:05:36.528722 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.528519    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-trusted-ca-bundle\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:05:36.528722 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:36.528527    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/040ebfaf-d53e-477a-81bf-9068decec17e-console-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:05:37.117148 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:37.117109    2580 generic.go:356] "Generic (PLEG): container finished" podID="040ebfaf-d53e-477a-81bf-9068decec17e" containerID="e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129" exitCode=0
Jul 15 21:05:37.117538 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:37.117162    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68794bd79c-88pvn"
Jul 15 21:05:37.117538 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:37.117201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68794bd79c-88pvn" event={"ID":"040ebfaf-d53e-477a-81bf-9068decec17e","Type":"ContainerDied","Data":"e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129"}
Jul 15 21:05:37.117538 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:37.117257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68794bd79c-88pvn" event={"ID":"040ebfaf-d53e-477a-81bf-9068decec17e","Type":"ContainerDied","Data":"9d35ac3497f771c6b4468d98ced9b700cacda953f56ed182cf6c1632fc23b1bf"}
Jul 15 21:05:37.117538 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:37.117279    2580 scope.go:117] "RemoveContainer" containerID="e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129"
Jul 15 21:05:37.125827 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:37.125806    2580 scope.go:117] "RemoveContainer" containerID="e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129"
Jul 15 21:05:37.126117 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:05:37.126095    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129\": container with ID starting with e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129 not found: ID does not exist" containerID="e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129"
Jul 15 21:05:37.126194 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:37.126130    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129"} err="failed to get container status \"e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129\": rpc error: code = NotFound desc = could not find container \"e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129\": container with ID starting with e3f84b75f5131caee4f58df78361665c2dd5de9867e14679bfc34b8d8df97129 not found: ID does not exist"
Jul 15 21:05:37.140280 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:37.140245    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68794bd79c-88pvn"]
Jul 15 21:05:37.147920 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:37.147896    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68794bd79c-88pvn"]
Jul 15 21:05:39.132602 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:05:39.132571    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="040ebfaf-d53e-477a-81bf-9068decec17e" path="/var/lib/kubelet/pods/040ebfaf-d53e-477a-81bf-9068decec17e/volumes"
Jul 15 21:06:07.285784 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.285748    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bc6685dc7-6lxmr"]
Jul 15 21:06:07.286284 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.286075    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="040ebfaf-d53e-477a-81bf-9068decec17e" containerName="console"
Jul 15 21:06:07.286284 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.286088    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="040ebfaf-d53e-477a-81bf-9068decec17e" containerName="console"
Jul 15 21:06:07.286284 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.286135    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="040ebfaf-d53e-477a-81bf-9068decec17e" containerName="console"
Jul 15 21:06:07.289000 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.288979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.300346 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.300317    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc6685dc7-6lxmr"]
Jul 15 21:06:07.383178 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.383137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-config\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.383178 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.383177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-trusted-ca-bundle\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.383432 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.383201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqkl8\" (UniqueName: \"kubernetes.io/projected/d385a56b-30dd-495c-8d41-adf04c1c0f2b-kube-api-access-zqkl8\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.383432 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.383301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-oauth-config\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.383432 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.383405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-serving-cert\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.383547 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.383443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-oauth-serving-cert\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.383547 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.383483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-service-ca\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.484901 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.484866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-serving-cert\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.484901 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.484904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-oauth-serving-cert\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.485173 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.484943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-service-ca\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.485173 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.485004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-config\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.485173 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.485031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-trusted-ca-bundle\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.485173 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.485061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqkl8\" (UniqueName: \"kubernetes.io/projected/d385a56b-30dd-495c-8d41-adf04c1c0f2b-kube-api-access-zqkl8\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.485173 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.485093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-oauth-config\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.485695 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.485671    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-oauth-serving-cert\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.485818 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.485796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-service-ca\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.485860 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.485796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-config\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.486117 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.486093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-trusted-ca-bundle\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.487419 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.487392    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-serving-cert\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.487540 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.487521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-oauth-config\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.493723 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.493692    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqkl8\" (UniqueName: \"kubernetes.io/projected/d385a56b-30dd-495c-8d41-adf04c1c0f2b-kube-api-access-zqkl8\") pod \"console-bc6685dc7-6lxmr\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") " pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.601173 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.601079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:07.730666 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:07.730643    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc6685dc7-6lxmr"]
Jul 15 21:06:07.733189 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:06:07.733162    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd385a56b_30dd_495c_8d41_adf04c1c0f2b.slice/crio-da218e532711e1d190327458354aa5148f64596cd9e677b9d64d13fa875d5daf WatchSource:0}: Error finding container da218e532711e1d190327458354aa5148f64596cd9e677b9d64d13fa875d5daf: Status 404 returned error can't find the container with id da218e532711e1d190327458354aa5148f64596cd9e677b9d64d13fa875d5daf
Jul 15 21:06:08.239488 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:08.239448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc6685dc7-6lxmr" event={"ID":"d385a56b-30dd-495c-8d41-adf04c1c0f2b","Type":"ContainerStarted","Data":"e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e"}
Jul 15 21:06:08.239682 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:08.239502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc6685dc7-6lxmr" event={"ID":"d385a56b-30dd-495c-8d41-adf04c1c0f2b","Type":"ContainerStarted","Data":"da218e532711e1d190327458354aa5148f64596cd9e677b9d64d13fa875d5daf"}
Jul 15 21:06:08.259119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:08.259052    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bc6685dc7-6lxmr" podStartSLOduration=1.259031684 podStartE2EDuration="1.259031684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:08.258818769 +0000 UTC m=+237.615737360" watchObservedRunningTime="2026-07-15 21:06:08.259031684 +0000 UTC m=+237.615950282"
Jul 15 21:06:17.601273 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:17.601211    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:17.601783 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:17.601302    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:17.606265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:17.606239    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:18.284854 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:18.284774    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:06:18.338730 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:18.338697    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54488fd664-gctvz"]
Jul 15 21:06:43.359750 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.359708    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54488fd664-gctvz" podUID="499d76ff-1f25-42c9-afcc-9ef76ea0f89a" containerName="console" containerID="cri-o://217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98" gracePeriod=15
Jul 15 21:06:43.599574 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.599547    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:06:43.669299 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669245    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7mpd\" (UniqueName: \"kubernetes.io/projected/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-kube-api-access-q7mpd\") pod \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") "
Jul 15 21:06:43.669299 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669281    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-trusted-ca-bundle\") pod \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") "
Jul 15 21:06:43.669421 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669311    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-service-ca\") pod \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") "
Jul 15 21:06:43.669501 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669481    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-config\") pod \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") "
Jul 15 21:06:43.669551 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669536    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-serving-cert\") pod \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") "
Jul 15 21:06:43.669613 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669599    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-oauth-serving-cert\") pod \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") "
Jul 15 21:06:43.669671 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669658    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-oauth-config\") pod \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\" (UID: \"499d76ff-1f25-42c9-afcc-9ef76ea0f89a\") "
Jul 15 21:06:43.669779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669664    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-service-ca" (OuterVolumeSpecName: "service-ca") pod "499d76ff-1f25-42c9-afcc-9ef76ea0f89a" (UID: "499d76ff-1f25-42c9-afcc-9ef76ea0f89a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:06:43.669779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669747    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "499d76ff-1f25-42c9-afcc-9ef76ea0f89a" (UID: "499d76ff-1f25-42c9-afcc-9ef76ea0f89a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:06:43.669915 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.669840    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-config" (OuterVolumeSpecName: "console-config") pod "499d76ff-1f25-42c9-afcc-9ef76ea0f89a" (UID: "499d76ff-1f25-42c9-afcc-9ef76ea0f89a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:06:43.670099 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.670077    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:06:43.670178 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.670104    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:06:43.670178 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.670115    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-trusted-ca-bundle\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:06:43.670178 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.670082    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "499d76ff-1f25-42c9-afcc-9ef76ea0f89a" (UID: "499d76ff-1f25-42c9-afcc-9ef76ea0f89a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:06:43.671389 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.671371    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-kube-api-access-q7mpd" (OuterVolumeSpecName: "kube-api-access-q7mpd") pod "499d76ff-1f25-42c9-afcc-9ef76ea0f89a" (UID: "499d76ff-1f25-42c9-afcc-9ef76ea0f89a"). InnerVolumeSpecName "kube-api-access-q7mpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:06:43.671715 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.671698    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "499d76ff-1f25-42c9-afcc-9ef76ea0f89a" (UID: "499d76ff-1f25-42c9-afcc-9ef76ea0f89a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:06:43.671826 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.671807    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "499d76ff-1f25-42c9-afcc-9ef76ea0f89a" (UID: "499d76ff-1f25-42c9-afcc-9ef76ea0f89a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:06:43.771314 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.771287    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7mpd\" (UniqueName: \"kubernetes.io/projected/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-kube-api-access-q7mpd\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:06:43.771314 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.771311    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:06:43.771441 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.771325    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-oauth-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:06:43.771441 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:43.771337    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/499d76ff-1f25-42c9-afcc-9ef76ea0f89a-console-oauth-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:06:44.370889 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:44.370855    2580 generic.go:356] "Generic (PLEG): container finished" podID="499d76ff-1f25-42c9-afcc-9ef76ea0f89a" containerID="217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98" exitCode=0
Jul 15 21:06:44.371365 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:44.370901    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54488fd664-gctvz"
Jul 15 21:06:44.371365 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:44.370940    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54488fd664-gctvz" event={"ID":"499d76ff-1f25-42c9-afcc-9ef76ea0f89a","Type":"ContainerDied","Data":"217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98"}
Jul 15 21:06:44.371365 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:44.371011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54488fd664-gctvz" event={"ID":"499d76ff-1f25-42c9-afcc-9ef76ea0f89a","Type":"ContainerDied","Data":"5dd6b359e830c54464e4ed9a8336fd3091bcb6aea3373008a0cdc61558cb8d65"}
Jul 15 21:06:44.371365 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:44.371036    2580 scope.go:117] "RemoveContainer" containerID="217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98"
Jul 15 21:06:44.379387 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:44.379371    2580 scope.go:117] "RemoveContainer" containerID="217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98"
Jul 15 21:06:44.379628 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:06:44.379610    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98\": container with ID starting with 217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98 not found: ID does not exist" containerID="217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98"
Jul 15 21:06:44.379670 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:44.379636    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98"} err="failed to get container status \"217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98\": rpc error: code = NotFound desc = could not find container \"217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98\": container with ID starting with 217f253e0293835329fe2aa785ec9bfb5e3aa7670644e672154e16f88c84df98 not found: ID does not exist"
Jul 15 21:06:44.393133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:44.393113    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54488fd664-gctvz"]
Jul 15 21:06:44.396865 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:44.396840    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54488fd664-gctvz"]
Jul 15 21:06:45.133019 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:06:45.132984    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="499d76ff-1f25-42c9-afcc-9ef76ea0f89a" path="/var/lib/kubelet/pods/499d76ff-1f25-42c9-afcc-9ef76ea0f89a/volumes"
Jul 15 21:07:11.002358 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:11.002328    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:07:11.003210 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:11.003188    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:07:11.006248 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:11.006222    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:07:11.006722 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:11.006702    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:07:11.012720 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:11.012704    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 21:07:30.060397 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.060357    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"]
Jul 15 21:07:30.062805 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.060746    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="499d76ff-1f25-42c9-afcc-9ef76ea0f89a" containerName="console"
Jul 15 21:07:30.062805 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.060760    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="499d76ff-1f25-42c9-afcc-9ef76ea0f89a" containerName="console"
Jul 15 21:07:30.062805 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.060830    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="499d76ff-1f25-42c9-afcc-9ef76ea0f89a" containerName="console"
Jul 15 21:07:30.063590 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.063574    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"
Jul 15 21:07:30.066207 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.066183    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 15 21:07:30.066207 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.066186    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 15 21:07:30.066466 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.066218    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-sdpxj\""
Jul 15 21:07:30.078006 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.077981    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"]
Jul 15 21:07:30.117412 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.117370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58q4m\" (UniqueName: \"kubernetes.io/projected/b67641ba-a0a6-4e1a-8086-962db85e8e51-kube-api-access-58q4m\") pod \"cert-manager-operator-controller-manager-55c444bf86-8zfvj\" (UID: \"b67641ba-a0a6-4e1a-8086-962db85e8e51\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"
Jul 15 21:07:30.117587 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.117431    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b67641ba-a0a6-4e1a-8086-962db85e8e51-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8zfvj\" (UID: \"b67641ba-a0a6-4e1a-8086-962db85e8e51\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"
Jul 15 21:07:30.218584 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.218551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b67641ba-a0a6-4e1a-8086-962db85e8e51-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8zfvj\" (UID: \"b67641ba-a0a6-4e1a-8086-962db85e8e51\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"
Jul 15 21:07:30.218741 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.218613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58q4m\" (UniqueName: \"kubernetes.io/projected/b67641ba-a0a6-4e1a-8086-962db85e8e51-kube-api-access-58q4m\") pod \"cert-manager-operator-controller-manager-55c444bf86-8zfvj\" (UID: \"b67641ba-a0a6-4e1a-8086-962db85e8e51\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"
Jul 15 21:07:30.218942 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.218921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b67641ba-a0a6-4e1a-8086-962db85e8e51-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8zfvj\" (UID: \"b67641ba-a0a6-4e1a-8086-962db85e8e51\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"
Jul 15 21:07:30.226442 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.226411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58q4m\" (UniqueName: \"kubernetes.io/projected/b67641ba-a0a6-4e1a-8086-962db85e8e51-kube-api-access-58q4m\") pod \"cert-manager-operator-controller-manager-55c444bf86-8zfvj\" (UID: \"b67641ba-a0a6-4e1a-8086-962db85e8e51\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"
Jul 15 21:07:30.372736 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.372654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"
Jul 15 21:07:30.524144 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:07:30.524112    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb67641ba_a0a6_4e1a_8086_962db85e8e51.slice/crio-a0e5c5606ceff5871c4d80d7df9750a56abaece7e0e061a8e0809bbe770ec0a5 WatchSource:0}: Error finding container a0e5c5606ceff5871c4d80d7df9750a56abaece7e0e061a8e0809bbe770ec0a5: Status 404 returned error can't find the container with id a0e5c5606ceff5871c4d80d7df9750a56abaece7e0e061a8e0809bbe770ec0a5
Jul 15 21:07:30.526944 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.526921    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:07:30.529314 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.529219    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj"]
Jul 15 21:07:30.544035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:30.544009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj" event={"ID":"b67641ba-a0a6-4e1a-8086-962db85e8e51","Type":"ContainerStarted","Data":"a0e5c5606ceff5871c4d80d7df9750a56abaece7e0e061a8e0809bbe770ec0a5"}
Jul 15 21:07:33.566643 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:33.566609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj" event={"ID":"b67641ba-a0a6-4e1a-8086-962db85e8e51","Type":"ContainerStarted","Data":"2d76e809f7b160b8c2c62739608264e381a5ec0cc917ddb1a7d36bba3bd5dda9"}
Jul 15 21:07:33.646635 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:33.646586    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8zfvj" podStartSLOduration=1.1655650180000001 podStartE2EDuration="3.646571194s" totalImagesPullingTime="2.481006176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:33.64550537 +0000 UTC m=+323.002423967" watchObservedRunningTime="2026-07-15 21:07:33.646571194 +0000 UTC m=+323.003489792"
Jul 15 21:07:34.025628 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.025545    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-lgh4r"]
Jul 15 21:07:34.028881 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.028861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:07:34.031719 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.031687    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 15 21:07:34.031838 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.031725    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-j424q\""
Jul 15 21:07:34.031838 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.031768    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 15 21:07:34.050753 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.050734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f139f8d2-e88e-407b-b300-af3dcbdc9ec8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-lgh4r\" (UID: \"f139f8d2-e88e-407b-b300-af3dcbdc9ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:07:34.050857 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.050837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khngj\" (UniqueName: \"kubernetes.io/projected/f139f8d2-e88e-407b-b300-af3dcbdc9ec8-kube-api-access-khngj\") pod \"cert-manager-webhook-6486ffd997-lgh4r\" (UID: \"f139f8d2-e88e-407b-b300-af3dcbdc9ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:07:34.051053 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.051037    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-lgh4r"]
Jul 15 21:07:34.151654 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.151627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khngj\" (UniqueName: \"kubernetes.io/projected/f139f8d2-e88e-407b-b300-af3dcbdc9ec8-kube-api-access-khngj\") pod \"cert-manager-webhook-6486ffd997-lgh4r\" (UID: \"f139f8d2-e88e-407b-b300-af3dcbdc9ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:07:34.151808 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.151663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f139f8d2-e88e-407b-b300-af3dcbdc9ec8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-lgh4r\" (UID: \"f139f8d2-e88e-407b-b300-af3dcbdc9ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:07:34.174780 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.174755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f139f8d2-e88e-407b-b300-af3dcbdc9ec8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-lgh4r\" (UID: \"f139f8d2-e88e-407b-b300-af3dcbdc9ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:07:34.174924 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.174779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khngj\" (UniqueName: \"kubernetes.io/projected/f139f8d2-e88e-407b-b300-af3dcbdc9ec8-kube-api-access-khngj\") pod \"cert-manager-webhook-6486ffd997-lgh4r\" (UID: \"f139f8d2-e88e-407b-b300-af3dcbdc9ec8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:07:34.358216 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.358176    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:07:34.546133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.546104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-lgh4r"]
Jul 15 21:07:34.549251 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:07:34.549221    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf139f8d2_e88e_407b_b300_af3dcbdc9ec8.slice/crio-44a0b75de2cbde606e13d0b3102c71cab04757c4524a351fec5c38e5a4962809 WatchSource:0}: Error finding container 44a0b75de2cbde606e13d0b3102c71cab04757c4524a351fec5c38e5a4962809: Status 404 returned error can't find the container with id 44a0b75de2cbde606e13d0b3102c71cab04757c4524a351fec5c38e5a4962809
Jul 15 21:07:34.571700 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:34.571673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r" event={"ID":"f139f8d2-e88e-407b-b300-af3dcbdc9ec8","Type":"ContainerStarted","Data":"44a0b75de2cbde606e13d0b3102c71cab04757c4524a351fec5c38e5a4962809"}
Jul 15 21:07:36.327731 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.327701    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4smql"]
Jul 15 21:07:36.331045 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.331022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql"
Jul 15 21:07:36.333532 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.333510    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-shgxr\""
Jul 15 21:07:36.342548 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.342526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4smql"]
Jul 15 21:07:36.369773 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.369749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb54b1b5-868a-4d33-ba4e-d75e364a7fd8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4smql\" (UID: \"eb54b1b5-868a-4d33-ba4e-d75e364a7fd8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql"
Jul 15 21:07:36.369907 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.369851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9b6w\" (UniqueName: \"kubernetes.io/projected/eb54b1b5-868a-4d33-ba4e-d75e364a7fd8-kube-api-access-q9b6w\") pod \"cert-manager-cainjector-559cdd8c87-4smql\" (UID: \"eb54b1b5-868a-4d33-ba4e-d75e364a7fd8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql"
Jul 15 21:07:36.470457 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.470422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9b6w\" (UniqueName: \"kubernetes.io/projected/eb54b1b5-868a-4d33-ba4e-d75e364a7fd8-kube-api-access-q9b6w\") pod \"cert-manager-cainjector-559cdd8c87-4smql\" (UID: \"eb54b1b5-868a-4d33-ba4e-d75e364a7fd8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql"
Jul 15 21:07:36.470647 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.470498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb54b1b5-868a-4d33-ba4e-d75e364a7fd8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4smql\" (UID: \"eb54b1b5-868a-4d33-ba4e-d75e364a7fd8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql"
Jul 15 21:07:36.480532 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.480507    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb54b1b5-868a-4d33-ba4e-d75e364a7fd8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4smql\" (UID: \"eb54b1b5-868a-4d33-ba4e-d75e364a7fd8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql"
Jul 15 21:07:36.480682 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.480650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9b6w\" (UniqueName: \"kubernetes.io/projected/eb54b1b5-868a-4d33-ba4e-d75e364a7fd8-kube-api-access-q9b6w\") pod \"cert-manager-cainjector-559cdd8c87-4smql\" (UID: \"eb54b1b5-868a-4d33-ba4e-d75e364a7fd8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql"
Jul 15 21:07:36.643163 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.643078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql"
Jul 15 21:07:36.788099 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:36.788072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4smql"]
Jul 15 21:07:37.634674 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:07:37.634642    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb54b1b5_868a_4d33_ba4e_d75e364a7fd8.slice/crio-a17f3abf693b341e0a0500aa7a9c5a8ac838d72c7f20a07719e817fcdfe4bb61 WatchSource:0}: Error finding container a17f3abf693b341e0a0500aa7a9c5a8ac838d72c7f20a07719e817fcdfe4bb61: Status 404 returned error can't find the container with id a17f3abf693b341e0a0500aa7a9c5a8ac838d72c7f20a07719e817fcdfe4bb61
Jul 15 21:07:38.590447 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:38.590411    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql" event={"ID":"eb54b1b5-868a-4d33-ba4e-d75e364a7fd8","Type":"ContainerStarted","Data":"dd4ecf20547e971a316ba624c4423f368f51556cac75ef19307a128aff88a8d3"}
Jul 15 21:07:38.590447 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:38.590459    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql" event={"ID":"eb54b1b5-868a-4d33-ba4e-d75e364a7fd8","Type":"ContainerStarted","Data":"a17f3abf693b341e0a0500aa7a9c5a8ac838d72c7f20a07719e817fcdfe4bb61"}
Jul 15 21:07:38.592035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:38.592009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r" event={"ID":"f139f8d2-e88e-407b-b300-af3dcbdc9ec8","Type":"ContainerStarted","Data":"c58042b186cc5b13ff533c1ecc1006c51319a38a09f1a4072e48e27dba36d7fd"}
Jul 15 21:07:38.592181 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:38.592170    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:07:38.607167 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:38.607124    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4smql" podStartSLOduration=2.060127056 podStartE2EDuration="2.607112019s" totalImagesPullingTime="546.984963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:38.606164178 +0000 UTC m=+327.963082775" watchObservedRunningTime="2026-07-15 21:07:38.607112019 +0000 UTC m=+327.964030616"
Jul 15 21:07:38.622077 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:38.622028    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r" podStartSLOduration=1.480809292 podStartE2EDuration="4.622009225s" totalImagesPullingTime="3.141199933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:38.621704271 +0000 UTC m=+327.978622867" watchObservedRunningTime="2026-07-15 21:07:38.622009225 +0000 UTC m=+327.978927821"
Jul 15 21:07:44.597803 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:07:44.597772    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-lgh4r"
Jul 15 21:08:07.117459 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.117384    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"]
Jul 15 21:08:07.120844 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.120825    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.133288 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.133265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 21:08:07.133511 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.133496    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 21:08:07.133596 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.133569    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cfr2c\""
Jul 15 21:08:07.133653 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.133594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 21:08:07.133700 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.133659    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 21:08:07.186436 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.186412    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"]
Jul 15 21:08:07.225710 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.225679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e37669e5-eb85-4e79-a7bc-83a947ca45b6-manifests\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.225832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.225803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e37669e5-eb85-4e79-a7bc-83a947ca45b6-tmp\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.225896 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.225848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e37669e5-eb85-4e79-a7bc-83a947ca45b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.226012 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.225989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5grhf\" (UniqueName: \"kubernetes.io/projected/e37669e5-eb85-4e79-a7bc-83a947ca45b6-kube-api-access-5grhf\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.226067 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.226048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e37669e5-eb85-4e79-a7bc-83a947ca45b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.327171 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.327145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e37669e5-eb85-4e79-a7bc-83a947ca45b6-manifests\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.327283 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.327179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e37669e5-eb85-4e79-a7bc-83a947ca45b6-tmp\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.327283 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.327199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e37669e5-eb85-4e79-a7bc-83a947ca45b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.327283 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.327221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5grhf\" (UniqueName: \"kubernetes.io/projected/e37669e5-eb85-4e79-a7bc-83a947ca45b6-kube-api-access-5grhf\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.327283 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.327251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e37669e5-eb85-4e79-a7bc-83a947ca45b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.327602 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.327581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e37669e5-eb85-4e79-a7bc-83a947ca45b6-manifests\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.329328 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.329307    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e37669e5-eb85-4e79-a7bc-83a947ca45b6-tmp\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.329673 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.329651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e37669e5-eb85-4e79-a7bc-83a947ca45b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.329729 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.329672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e37669e5-eb85-4e79-a7bc-83a947ca45b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.335804 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.335784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5grhf\" (UniqueName: \"kubernetes.io/projected/e37669e5-eb85-4e79-a7bc-83a947ca45b6-kube-api-access-5grhf\") pod \"opendatahub-operator-controller-manager-65c67f598f-qjkkv\" (UID: \"e37669e5-eb85-4e79-a7bc-83a947ca45b6\") " pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.430618 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.430564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:07.567203 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.567171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"]
Jul 15 21:08:07.570768 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:08:07.570743    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37669e5_eb85_4e79_a7bc_83a947ca45b6.slice/crio-f6c1a230bfc5b1c915f97dc71e644f448e5f2db06435cc97ac4c62a9514f9cae WatchSource:0}: Error finding container f6c1a230bfc5b1c915f97dc71e644f448e5f2db06435cc97ac4c62a9514f9cae: Status 404 returned error can't find the container with id f6c1a230bfc5b1c915f97dc71e644f448e5f2db06435cc97ac4c62a9514f9cae
Jul 15 21:08:07.719186 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:07.719117    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv" event={"ID":"e37669e5-eb85-4e79-a7bc-83a947ca45b6","Type":"ContainerStarted","Data":"f6c1a230bfc5b1c915f97dc71e644f448e5f2db06435cc97ac4c62a9514f9cae"}
Jul 15 21:08:10.734755 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:10.734719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv" event={"ID":"e37669e5-eb85-4e79-a7bc-83a947ca45b6","Type":"ContainerStarted","Data":"494346fdf0652075c4e0dff5d81acdc58e4e7ebe9a3522611e7e628860bb6d65"}
Jul 15 21:08:11.741148 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:11.741109    2580 generic.go:356] "Generic (PLEG): container finished" podID="e37669e5-eb85-4e79-a7bc-83a947ca45b6" containerID="494346fdf0652075c4e0dff5d81acdc58e4e7ebe9a3522611e7e628860bb6d65" exitCode=0
Jul 15 21:08:11.741612 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:11.741179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv" event={"ID":"e37669e5-eb85-4e79-a7bc-83a947ca45b6","Type":"ContainerDied","Data":"494346fdf0652075c4e0dff5d81acdc58e4e7ebe9a3522611e7e628860bb6d65"}
Jul 15 21:08:12.747562 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:12.747522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv" event={"ID":"e37669e5-eb85-4e79-a7bc-83a947ca45b6","Type":"ContainerStarted","Data":"0142dcf9c68b1368e483ddb128788b7eb289fb9f07f798890e0915a4917acc02"}
Jul 15 21:08:12.748066 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:12.747644    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:12.771198 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:12.771142    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv" podStartSLOduration=1.317779268 podStartE2EDuration="5.771120997s" totalImagesPullingTime="3.410917506s" totalInitContainerRuntime="1.042424223s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:12.769890182 +0000 UTC m=+362.126808779" watchObservedRunningTime="2026-07-15 21:08:12.771120997 +0000 UTC m=+362.128039594"
Jul 15 21:08:23.755667 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:23.755635    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65c67f598f-qjkkv"
Jul 15 21:08:26.924365 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:26.924326    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-tqlcz"]
Jul 15 21:08:26.932794 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:26.932766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:26.935107 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:26.935083    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 15 21:08:26.940405 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:26.940379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-tqlcz"]
Jul 15 21:08:26.981384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:26.981357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9357ba15-5356-44b6-9451-d10b7c9f0616-tls-certs\") pod \"model-serving-api-54d89bd5fc-tqlcz\" (UID: \"9357ba15-5356-44b6-9451-d10b7c9f0616\") " pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:26.981532 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:26.981402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvvdb\" (UniqueName: \"kubernetes.io/projected/9357ba15-5356-44b6-9451-d10b7c9f0616-kube-api-access-rvvdb\") pod \"model-serving-api-54d89bd5fc-tqlcz\" (UID: \"9357ba15-5356-44b6-9451-d10b7c9f0616\") " pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:27.082728 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.082697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9357ba15-5356-44b6-9451-d10b7c9f0616-tls-certs\") pod \"model-serving-api-54d89bd5fc-tqlcz\" (UID: \"9357ba15-5356-44b6-9451-d10b7c9f0616\") " pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:27.082889 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.082762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvvdb\" (UniqueName: \"kubernetes.io/projected/9357ba15-5356-44b6-9451-d10b7c9f0616-kube-api-access-rvvdb\") pod \"model-serving-api-54d89bd5fc-tqlcz\" (UID: \"9357ba15-5356-44b6-9451-d10b7c9f0616\") " pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:27.082889 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:08:27.082845    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 15 21:08:27.083007 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:08:27.082917    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9357ba15-5356-44b6-9451-d10b7c9f0616-tls-certs podName:9357ba15-5356-44b6-9451-d10b7c9f0616 nodeName:}" failed. No retries permitted until 2026-07-15 21:08:27.582901513 +0000 UTC m=+376.939820088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9357ba15-5356-44b6-9451-d10b7c9f0616-tls-certs") pod "model-serving-api-54d89bd5fc-tqlcz" (UID: "9357ba15-5356-44b6-9451-d10b7c9f0616") : secret "model-serving-api-tls" not found
Jul 15 21:08:27.094606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.094582    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"]
Jul 15 21:08:27.097678 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.097654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.099881 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.099856    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 21:08:27.100011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.099856    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 21:08:27.100011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.099919    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 21:08:27.100125 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.100050    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 21:08:27.100125 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.100056    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r52kh\""
Jul 15 21:08:27.100475 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.100451    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 21:08:27.100815 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.100797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvvdb\" (UniqueName: \"kubernetes.io/projected/9357ba15-5356-44b6-9451-d10b7c9f0616-kube-api-access-rvvdb\") pod \"model-serving-api-54d89bd5fc-tqlcz\" (UID: \"9357ba15-5356-44b6-9451-d10b7c9f0616\") " pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:27.111267 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.111245    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"]
Jul 15 21:08:27.183158 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.183079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-manager-config\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.183158 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.183131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-metrics-cert\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.183362 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.183223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-cert\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.183362 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.183281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrb56\" (UniqueName: \"kubernetes.io/projected/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-kube-api-access-wrb56\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.284267 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.284239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-metrics-cert\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.284473 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.284290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-cert\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.284473 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.284329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrb56\" (UniqueName: \"kubernetes.io/projected/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-kube-api-access-wrb56\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.284473 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.284350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-manager-config\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.285095 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.285062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-manager-config\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.286840 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.286821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-cert\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.286931 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.286850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-metrics-cert\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.292607 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.292582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrb56\" (UniqueName: \"kubernetes.io/projected/f9f252d2-aa4b-46fd-b0ae-8bf74c29513c-kube-api-access-wrb56\") pod \"lws-controller-manager-5ccd478559-n6rwj\" (UID: \"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c\") " pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.414782 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.414751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:27.586383 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.586346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9357ba15-5356-44b6-9451-d10b7c9f0616-tls-certs\") pod \"model-serving-api-54d89bd5fc-tqlcz\" (UID: \"9357ba15-5356-44b6-9451-d10b7c9f0616\") " pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:27.586565 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:08:27.586494    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 15 21:08:27.586618 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:08:27.586568    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9357ba15-5356-44b6-9451-d10b7c9f0616-tls-certs podName:9357ba15-5356-44b6-9451-d10b7c9f0616 nodeName:}" failed. No retries permitted until 2026-07-15 21:08:28.586554005 +0000 UTC m=+377.943472583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9357ba15-5356-44b6-9451-d10b7c9f0616-tls-certs") pod "model-serving-api-54d89bd5fc-tqlcz" (UID: "9357ba15-5356-44b6-9451-d10b7c9f0616") : secret "model-serving-api-tls" not found
Jul 15 21:08:27.596505 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.596486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"]
Jul 15 21:08:27.599144 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:08:27.599117    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f252d2_aa4b_46fd_b0ae_8bf74c29513c.slice/crio-f3a84551c6c331355a63bfc4f41b2964341fab6edf1dce26b6ea303c8f354626 WatchSource:0}: Error finding container f3a84551c6c331355a63bfc4f41b2964341fab6edf1dce26b6ea303c8f354626: Status 404 returned error can't find the container with id f3a84551c6c331355a63bfc4f41b2964341fab6edf1dce26b6ea303c8f354626
Jul 15 21:08:27.812823 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:27.812788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj" event={"ID":"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c","Type":"ContainerStarted","Data":"f3a84551c6c331355a63bfc4f41b2964341fab6edf1dce26b6ea303c8f354626"}
Jul 15 21:08:28.599091 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:28.599050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9357ba15-5356-44b6-9451-d10b7c9f0616-tls-certs\") pod \"model-serving-api-54d89bd5fc-tqlcz\" (UID: \"9357ba15-5356-44b6-9451-d10b7c9f0616\") " pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:28.601975 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:28.601931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9357ba15-5356-44b6-9451-d10b7c9f0616-tls-certs\") pod \"model-serving-api-54d89bd5fc-tqlcz\" (UID: \"9357ba15-5356-44b6-9451-d10b7c9f0616\") " pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:28.745153 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:28.745105    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:28.900145 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:28.900066    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-tqlcz"]
Jul 15 21:08:28.904516 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:08:28.903950    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9357ba15_5356_44b6_9451_d10b7c9f0616.slice/crio-5ed29a515fc1e2f4ef7f12b0fe27c90e51337784043e519855ef806a3a1b9f38 WatchSource:0}: Error finding container 5ed29a515fc1e2f4ef7f12b0fe27c90e51337784043e519855ef806a3a1b9f38: Status 404 returned error can't find the container with id 5ed29a515fc1e2f4ef7f12b0fe27c90e51337784043e519855ef806a3a1b9f38
Jul 15 21:08:29.823062 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:29.823020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz" event={"ID":"9357ba15-5356-44b6-9451-d10b7c9f0616","Type":"ContainerStarted","Data":"5ed29a515fc1e2f4ef7f12b0fe27c90e51337784043e519855ef806a3a1b9f38"}
Jul 15 21:08:31.838394 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:31.838353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj" event={"ID":"f9f252d2-aa4b-46fd-b0ae-8bf74c29513c","Type":"ContainerStarted","Data":"80372e89ae953bb0b59b7b257028214041ce825e857cbff1ce72d5b5886850a4"}
Jul 15 21:08:31.838853 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:31.838448    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:31.839858 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:31.839831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz" event={"ID":"9357ba15-5356-44b6-9451-d10b7c9f0616","Type":"ContainerStarted","Data":"ce972474d37e4ba965693b5122ca752eb81ae80749216021c177441bc9b74edf"}
Jul 15 21:08:31.839996 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:31.839981    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:31.855572 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:31.855531    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj" podStartSLOduration=1.096766458 podStartE2EDuration="4.855518417s" totalImagesPullingTime="3.758751959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:31.853914703 +0000 UTC m=+381.210833299" watchObservedRunningTime="2026-07-15 21:08:31.855518417 +0000 UTC m=+381.212437015"
Jul 15 21:08:31.872026 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:31.871982    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz" podStartSLOduration=3.419884673 podStartE2EDuration="5.871950373s" totalImagesPullingTime="2.4520657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:31.871127701 +0000 UTC m=+381.228046319" watchObservedRunningTime="2026-07-15 21:08:31.871950373 +0000 UTC m=+381.228868970"
Jul 15 21:08:34.091167 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.091123    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"]
Jul 15 21:08:34.094703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.094681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.098385 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.098363    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 15 21:08:34.098642 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.098621    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mj8wt\""
Jul 15 21:08:34.099383 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.099366    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 15 21:08:34.099491 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.099397    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 21:08:34.099491 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.099431    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 21:08:34.099606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.099401    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 15 21:08:34.099606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.099502    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 21:08:34.120727 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.120698    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"]
Jul 15 21:08:34.153566 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.153531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.153566 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.153569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.153756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.153601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.153756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.153628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26lwd\" (UniqueName: \"kubernetes.io/projected/dee496e1-3e51-4ff6-85fb-9058fcf1d770-kube-api-access-26lwd\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.153756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.153649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dee496e1-3e51-4ff6-85fb-9058fcf1d770-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.153756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.153667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee496e1-3e51-4ff6-85fb-9058fcf1d770-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.153756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.153731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.254172 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.254127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26lwd\" (UniqueName: \"kubernetes.io/projected/dee496e1-3e51-4ff6-85fb-9058fcf1d770-kube-api-access-26lwd\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.254172 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.254178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dee496e1-3e51-4ff6-85fb-9058fcf1d770-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.254433 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.254204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee496e1-3e51-4ff6-85fb-9058fcf1d770-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.254433 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.254249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.254433 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.254283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.254433 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.254301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.254433 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.254344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.254936 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.254909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.256658 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.256633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.256748 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.256717    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dee496e1-3e51-4ff6-85fb-9058fcf1d770-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.256876 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.256860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee496e1-3e51-4ff6-85fb-9058fcf1d770-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.257035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.257019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.262666 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.262642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26lwd\" (UniqueName: \"kubernetes.io/projected/dee496e1-3e51-4ff6-85fb-9058fcf1d770-kube-api-access-26lwd\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.262857 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.262838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee496e1-3e51-4ff6-85fb-9058fcf1d770-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xvvsh\" (UID: \"dee496e1-3e51-4ff6-85fb-9058fcf1d770\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.406242 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.406151    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:34.545688 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.545660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"]
Jul 15 21:08:34.547606 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:08:34.547574    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee496e1_3e51_4ff6_85fb_9058fcf1d770.slice/crio-50cd3add2fb59fb86020ca471939a73f4f05dad0323782344c08f7c464d0a2be WatchSource:0}: Error finding container 50cd3add2fb59fb86020ca471939a73f4f05dad0323782344c08f7c464d0a2be: Status 404 returned error can't find the container with id 50cd3add2fb59fb86020ca471939a73f4f05dad0323782344c08f7c464d0a2be
Jul 15 21:08:34.859660 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:34.859625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh" event={"ID":"dee496e1-3e51-4ff6-85fb-9058fcf1d770","Type":"ContainerStarted","Data":"50cd3add2fb59fb86020ca471939a73f4f05dad0323782344c08f7c464d0a2be"}
Jul 15 21:08:42.852409 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:42.852380    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5ccd478559-n6rwj"
Jul 15 21:08:42.854492 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:42.854470    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-tqlcz"
Jul 15 21:08:51.631912 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:51.631864    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Jul 15 21:08:51.632248 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:51.631982    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Jul 15 21:08:51.941604 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:51.941515    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh" event={"ID":"dee496e1-3e51-4ff6-85fb-9058fcf1d770","Type":"ContainerStarted","Data":"6f7db492962dfd4dac21323fbe7473b4df7ab25441946ec8f75885159aa1b4aa"}
Jul 15 21:08:51.941755 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:51.941629    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:08:51.961015 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:51.960944    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh" podStartSLOduration=0.879139982 podStartE2EDuration="17.960929198s" totalImagesPullingTime="17.081789216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:51.960011042 +0000 UTC m=+401.316929641" watchObservedRunningTime="2026-07-15 21:08:51.960929198 +0000 UTC m=+401.317847794"
Jul 15 21:08:52.947734 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:08:52.947700    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xvvsh"
Jul 15 21:10:07.804773 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.804740    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-777b79dc58-2pgbr"]
Jul 15 21:10:07.807909 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.807891    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.837760 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.837732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-service-ca\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.837879 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.837779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11473bef-cb2a-4508-96cd-9163714f1790-console-oauth-config\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.837879 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.837818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-trusted-ca-bundle\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.837986 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.837881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-oauth-serving-cert\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.837986 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.837909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-console-config\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.837986 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.837928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11473bef-cb2a-4508-96cd-9163714f1790-console-serving-cert\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.838098 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.838021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8rr5\" (UniqueName: \"kubernetes.io/projected/11473bef-cb2a-4508-96cd-9163714f1790-kube-api-access-f8rr5\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.938716 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.938679    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777b79dc58-2pgbr"]
Jul 15 21:10:07.938851 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.938806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-oauth-serving-cert\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.938851 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.938846    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-console-config\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.938929 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.938865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11473bef-cb2a-4508-96cd-9163714f1790-console-serving-cert\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.938929 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.938906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8rr5\" (UniqueName: \"kubernetes.io/projected/11473bef-cb2a-4508-96cd-9163714f1790-kube-api-access-f8rr5\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.939046 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.938933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-service-ca\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.939046 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.938991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11473bef-cb2a-4508-96cd-9163714f1790-console-oauth-config\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.939046 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.939008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-trusted-ca-bundle\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.939682 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.939657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-oauth-serving-cert\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.939830 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.939805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-service-ca\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.939990 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.939686    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-console-config\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.940074 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.939851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11473bef-cb2a-4508-96cd-9163714f1790-trusted-ca-bundle\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.941512 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.941489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11473bef-cb2a-4508-96cd-9163714f1790-console-serving-cert\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.941706 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.941688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11473bef-cb2a-4508-96cd-9163714f1790-console-oauth-config\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:07.983026 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:07.983000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8rr5\" (UniqueName: \"kubernetes.io/projected/11473bef-cb2a-4508-96cd-9163714f1790-kube-api-access-f8rr5\") pod \"console-777b79dc58-2pgbr\" (UID: \"11473bef-cb2a-4508-96cd-9163714f1790\") " pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:08.116937 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:08.116876    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:08.310337 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:08.310311    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777b79dc58-2pgbr"]
Jul 15 21:10:08.310703 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:10:08.310682    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11473bef_cb2a_4508_96cd_9163714f1790.slice/crio-9b370c4555c2c14d744080f4a3b9bbaeb7c2b60459bd2beac6b0f244edc87878 WatchSource:0}: Error finding container 9b370c4555c2c14d744080f4a3b9bbaeb7c2b60459bd2beac6b0f244edc87878: Status 404 returned error can't find the container with id 9b370c4555c2c14d744080f4a3b9bbaeb7c2b60459bd2beac6b0f244edc87878
Jul 15 21:10:09.289046 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:09.289011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777b79dc58-2pgbr" event={"ID":"11473bef-cb2a-4508-96cd-9163714f1790","Type":"ContainerStarted","Data":"0eea1c640a42797b03ceedfa4cc2dd0c7401906be779a9865db9b7eaa4d62ff0"}
Jul 15 21:10:09.289474 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:09.289068    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777b79dc58-2pgbr" event={"ID":"11473bef-cb2a-4508-96cd-9163714f1790","Type":"ContainerStarted","Data":"9b370c4555c2c14d744080f4a3b9bbaeb7c2b60459bd2beac6b0f244edc87878"}
Jul 15 21:10:09.341684 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:09.341630    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-777b79dc58-2pgbr" podStartSLOduration=2.341615495 podStartE2EDuration="2.341615495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:10:09.33962017 +0000 UTC m=+478.696538767" watchObservedRunningTime="2026-07-15 21:10:09.341615495 +0000 UTC m=+478.698534087"
Jul 15 21:10:18.117220 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:18.117181    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:18.117668 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:18.117245    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:18.121938 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:18.121916    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:18.334382 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:18.334357    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-777b79dc58-2pgbr"
Jul 15 21:10:18.488180 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:18.488094    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bc6685dc7-6lxmr"]
Jul 15 21:10:43.507511 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.507467    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bc6685dc7-6lxmr" podUID="d385a56b-30dd-495c-8d41-adf04c1c0f2b" containerName="console" containerID="cri-o://e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e" gracePeriod=15
Jul 15 21:10:43.744053 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.744028    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:10:43.823983 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.823946    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqkl8\" (UniqueName: \"kubernetes.io/projected/d385a56b-30dd-495c-8d41-adf04c1c0f2b-kube-api-access-zqkl8\") pod \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") "
Jul 15 21:10:43.824121 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824001    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-serving-cert\") pod \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") "
Jul 15 21:10:43.824121 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824018    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-trusted-ca-bundle\") pod \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") "
Jul 15 21:10:43.824237 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824133    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-oauth-config\") pod \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") "
Jul 15 21:10:43.824237 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824175    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-oauth-serving-cert\") pod \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") "
Jul 15 21:10:43.824237 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824204    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-service-ca\") pod \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") "
Jul 15 21:10:43.824377 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824263    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-config\") pod \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\" (UID: \"d385a56b-30dd-495c-8d41-adf04c1c0f2b\") "
Jul 15 21:10:43.824485 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824459    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d385a56b-30dd-495c-8d41-adf04c1c0f2b" (UID: "d385a56b-30dd-495c-8d41-adf04c1c0f2b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:10:43.824588 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824569    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-trusted-ca-bundle\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:10:43.824588 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824569    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d385a56b-30dd-495c-8d41-adf04c1c0f2b" (UID: "d385a56b-30dd-495c-8d41-adf04c1c0f2b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:10:43.824730 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824671    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-service-ca" (OuterVolumeSpecName: "service-ca") pod "d385a56b-30dd-495c-8d41-adf04c1c0f2b" (UID: "d385a56b-30dd-495c-8d41-adf04c1c0f2b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:10:43.824778 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.824743    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-config" (OuterVolumeSpecName: "console-config") pod "d385a56b-30dd-495c-8d41-adf04c1c0f2b" (UID: "d385a56b-30dd-495c-8d41-adf04c1c0f2b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:10:43.826110 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.826091    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d385a56b-30dd-495c-8d41-adf04c1c0f2b" (UID: "d385a56b-30dd-495c-8d41-adf04c1c0f2b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:10:43.826486 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.826468    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d385a56b-30dd-495c-8d41-adf04c1c0f2b-kube-api-access-zqkl8" (OuterVolumeSpecName: "kube-api-access-zqkl8") pod "d385a56b-30dd-495c-8d41-adf04c1c0f2b" (UID: "d385a56b-30dd-495c-8d41-adf04c1c0f2b"). InnerVolumeSpecName "kube-api-access-zqkl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:10:43.826565 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.826473    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d385a56b-30dd-495c-8d41-adf04c1c0f2b" (UID: "d385a56b-30dd-495c-8d41-adf04c1c0f2b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:10:43.925473 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.925451    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-oauth-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:10:43.925473 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.925471    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:10:43.925589 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.925479    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:10:43.925589 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.925487    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqkl8\" (UniqueName: \"kubernetes.io/projected/d385a56b-30dd-495c-8d41-adf04c1c0f2b-kube-api-access-zqkl8\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:10:43.925589 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.925495    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-serving-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:10:43.925589 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:43.925504    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d385a56b-30dd-495c-8d41-adf04c1c0f2b-console-oauth-config\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:10:44.444811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:44.444778    2580 generic.go:356] "Generic (PLEG): container finished" podID="d385a56b-30dd-495c-8d41-adf04c1c0f2b" containerID="e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e" exitCode=0
Jul 15 21:10:44.445011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:44.444834    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc6685dc7-6lxmr"
Jul 15 21:10:44.445011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:44.444850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc6685dc7-6lxmr" event={"ID":"d385a56b-30dd-495c-8d41-adf04c1c0f2b","Type":"ContainerDied","Data":"e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e"}
Jul 15 21:10:44.445011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:44.444892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc6685dc7-6lxmr" event={"ID":"d385a56b-30dd-495c-8d41-adf04c1c0f2b","Type":"ContainerDied","Data":"da218e532711e1d190327458354aa5148f64596cd9e677b9d64d13fa875d5daf"}
Jul 15 21:10:44.445011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:44.444912    2580 scope.go:117] "RemoveContainer" containerID="e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e"
Jul 15 21:10:44.455134 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:44.455115    2580 scope.go:117] "RemoveContainer" containerID="e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e"
Jul 15 21:10:44.461922 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:10:44.461892    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e\": container with ID starting with e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e not found: ID does not exist" containerID="e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e"
Jul 15 21:10:44.462045 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:44.461928    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e"} err="failed to get container status \"e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e\": rpc error: code = NotFound desc = could not find container \"e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e\": container with ID starting with e8c0c18a94d222a5ccae02ebc028872a3dd40ff1124d06d51bd3a2a86fa4c32e not found: ID does not exist"
Jul 15 21:10:44.475252 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:44.475232    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bc6685dc7-6lxmr"]
Jul 15 21:10:44.483450 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:44.483409    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bc6685dc7-6lxmr"]
Jul 15 21:10:45.134389 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:10:45.134354    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d385a56b-30dd-495c-8d41-adf04c1c0f2b" path="/var/lib/kubelet/pods/d385a56b-30dd-495c-8d41-adf04c1c0f2b/volumes"
Jul 15 21:11:27.380244 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.380211    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b868d9f9-5dwrs"]
Jul 15 21:11:27.380691 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.380539    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d385a56b-30dd-495c-8d41-adf04c1c0f2b" containerName="console"
Jul 15 21:11:27.380691 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.380564    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d385a56b-30dd-495c-8d41-adf04c1c0f2b" containerName="console"
Jul 15 21:11:27.380691 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.380636    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d385a56b-30dd-495c-8d41-adf04c1c0f2b" containerName="console"
Jul 15 21:11:27.384751 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.384735    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:27.386840 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.386815    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7f6jx\""
Jul 15 21:11:27.387105 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.387088    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 21:11:27.387690 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.387679    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 21:11:27.391272 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.391251    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b868d9f9-5dwrs"]
Jul 15 21:11:27.449140 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.449111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcs5q\" (UniqueName: \"kubernetes.io/projected/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-kube-api-access-mcs5q\") pod \"maas-controller-6b868d9f9-5dwrs\" (UID: \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\") " pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:27.449273 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.449144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-webhook-cert\") pod \"maas-controller-6b868d9f9-5dwrs\" (UID: \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\") " pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:27.549722 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.549697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcs5q\" (UniqueName: \"kubernetes.io/projected/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-kube-api-access-mcs5q\") pod \"maas-controller-6b868d9f9-5dwrs\" (UID: \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\") " pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:27.549844 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.549728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-webhook-cert\") pod \"maas-controller-6b868d9f9-5dwrs\" (UID: \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\") " pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:27.549844 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:11:27.549828    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 15 21:11:27.549924 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:11:27.549884    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-webhook-cert podName:266d8ee2-7184-4f2f-b1dd-cdea44709d9f nodeName:}" failed. No retries permitted until 2026-07-15 21:11:28.049869192 +0000 UTC m=+557.406787767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-webhook-cert") pod "maas-controller-6b868d9f9-5dwrs" (UID: "266d8ee2-7184-4f2f-b1dd-cdea44709d9f") : secret "maas-controller-webhook-cert" not found
Jul 15 21:11:27.562351 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.562326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcs5q\" (UniqueName: \"kubernetes.io/projected/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-kube-api-access-mcs5q\") pod \"maas-controller-6b868d9f9-5dwrs\" (UID: \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\") " pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:27.562782 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.562761    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b45b4f795-k985s"]
Jul 15 21:11:27.565994 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.565978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:27.578094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.578073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b45b4f795-k985s"]
Jul 15 21:11:27.651195 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.651121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1eda09d-f843-405f-bfb3-ee9356ab5246-webhook-cert\") pod \"maas-controller-6b45b4f795-k985s\" (UID: \"c1eda09d-f843-405f-bfb3-ee9356ab5246\") " pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:27.651195 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.651193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9flk\" (UniqueName: \"kubernetes.io/projected/c1eda09d-f843-405f-bfb3-ee9356ab5246-kube-api-access-g9flk\") pod \"maas-controller-6b45b4f795-k985s\" (UID: \"c1eda09d-f843-405f-bfb3-ee9356ab5246\") " pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:27.751929 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.751898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9flk\" (UniqueName: \"kubernetes.io/projected/c1eda09d-f843-405f-bfb3-ee9356ab5246-kube-api-access-g9flk\") pod \"maas-controller-6b45b4f795-k985s\" (UID: \"c1eda09d-f843-405f-bfb3-ee9356ab5246\") " pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:27.752061 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.751952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1eda09d-f843-405f-bfb3-ee9356ab5246-webhook-cert\") pod \"maas-controller-6b45b4f795-k985s\" (UID: \"c1eda09d-f843-405f-bfb3-ee9356ab5246\") " pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:27.754288 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.754266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1eda09d-f843-405f-bfb3-ee9356ab5246-webhook-cert\") pod \"maas-controller-6b45b4f795-k985s\" (UID: \"c1eda09d-f843-405f-bfb3-ee9356ab5246\") " pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:27.759854 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.759827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9flk\" (UniqueName: \"kubernetes.io/projected/c1eda09d-f843-405f-bfb3-ee9356ab5246-kube-api-access-g9flk\") pod \"maas-controller-6b45b4f795-k985s\" (UID: \"c1eda09d-f843-405f-bfb3-ee9356ab5246\") " pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:27.879746 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:27.879721    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:28.008839 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:28.008811    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b45b4f795-k985s"]
Jul 15 21:11:28.010560 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:11:28.010532    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1eda09d_f843_405f_bfb3_ee9356ab5246.slice/crio-f094fcec8fe6792fab61415c13f5913cf3fdf514d2fbc7100f81427f5b360146 WatchSource:0}: Error finding container f094fcec8fe6792fab61415c13f5913cf3fdf514d2fbc7100f81427f5b360146: Status 404 returned error can't find the container with id f094fcec8fe6792fab61415c13f5913cf3fdf514d2fbc7100f81427f5b360146
Jul 15 21:11:28.053989 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:28.053952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-webhook-cert\") pod \"maas-controller-6b868d9f9-5dwrs\" (UID: \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\") " pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:28.056128 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:28.056107    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-webhook-cert\") pod \"maas-controller-6b868d9f9-5dwrs\" (UID: \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\") " pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:28.295904 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:28.295876    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:28.425615 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:28.425589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b868d9f9-5dwrs"]
Jul 15 21:11:28.427826 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:11:28.427798    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266d8ee2_7184_4f2f_b1dd_cdea44709d9f.slice/crio-005ab352412c54cd748050dad567da6cccb77d051fc326ea3cd76fcf7b79faec WatchSource:0}: Error finding container 005ab352412c54cd748050dad567da6cccb77d051fc326ea3cd76fcf7b79faec: Status 404 returned error can't find the container with id 005ab352412c54cd748050dad567da6cccb77d051fc326ea3cd76fcf7b79faec
Jul 15 21:11:28.647900 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:28.647813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b45b4f795-k985s" event={"ID":"c1eda09d-f843-405f-bfb3-ee9356ab5246","Type":"ContainerStarted","Data":"f094fcec8fe6792fab61415c13f5913cf3fdf514d2fbc7100f81427f5b360146"}
Jul 15 21:11:28.649108 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:28.649070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b868d9f9-5dwrs" event={"ID":"266d8ee2-7184-4f2f-b1dd-cdea44709d9f","Type":"ContainerStarted","Data":"005ab352412c54cd748050dad567da6cccb77d051fc326ea3cd76fcf7b79faec"}
Jul 15 21:11:30.660153 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:30.660113    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b45b4f795-k985s" event={"ID":"c1eda09d-f843-405f-bfb3-ee9356ab5246","Type":"ContainerStarted","Data":"51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7"}
Jul 15 21:11:30.660564 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:30.660283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:30.661638 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:30.661612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b868d9f9-5dwrs" event={"ID":"266d8ee2-7184-4f2f-b1dd-cdea44709d9f","Type":"ContainerStarted","Data":"f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875"}
Jul 15 21:11:30.661764 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:30.661716    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:30.679648 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:30.679604    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b45b4f795-k985s" podStartSLOduration=1.147707082 podStartE2EDuration="3.679589904s" totalImagesPullingTime="2.531882822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:11:30.678265184 +0000 UTC m=+560.035183781" watchObservedRunningTime="2026-07-15 21:11:30.679589904 +0000 UTC m=+560.036508528"
Jul 15 21:11:30.699928 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:30.699888    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b868d9f9-5dwrs" podStartSLOduration=1.5787965640000001 podStartE2EDuration="3.699874779s" totalImagesPullingTime="2.121078215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:11:30.698114307 +0000 UTC m=+560.055032904" watchObservedRunningTime="2026-07-15 21:11:30.699874779 +0000 UTC m=+560.056793378"
Jul 15 21:11:41.668120 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:41.668085    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:41.668563 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:41.668253    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:11:41.749484 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:41.749452    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b868d9f9-5dwrs"]
Jul 15 21:11:41.749656 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:41.749631    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b868d9f9-5dwrs" podUID="266d8ee2-7184-4f2f-b1dd-cdea44709d9f" containerName="manager" containerID="cri-o://f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875" gracePeriod=30
Jul 15 21:11:41.986687 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:41.986665    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:42.066685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.066662    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcs5q\" (UniqueName: \"kubernetes.io/projected/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-kube-api-access-mcs5q\") pod \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\" (UID: \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\") "
Jul 15 21:11:42.066794 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.066723    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-webhook-cert\") pod \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\" (UID: \"266d8ee2-7184-4f2f-b1dd-cdea44709d9f\") "
Jul 15 21:11:42.068766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.068744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-kube-api-access-mcs5q" (OuterVolumeSpecName: "kube-api-access-mcs5q") pod "266d8ee2-7184-4f2f-b1dd-cdea44709d9f" (UID: "266d8ee2-7184-4f2f-b1dd-cdea44709d9f"). InnerVolumeSpecName "kube-api-access-mcs5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:11:42.068844 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.068825    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "266d8ee2-7184-4f2f-b1dd-cdea44709d9f" (UID: "266d8ee2-7184-4f2f-b1dd-cdea44709d9f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:11:42.167426 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.167400    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcs5q\" (UniqueName: \"kubernetes.io/projected/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-kube-api-access-mcs5q\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:11:42.167426 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.167425    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/266d8ee2-7184-4f2f-b1dd-cdea44709d9f-webhook-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:11:42.718453 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.718419    2580 generic.go:356] "Generic (PLEG): container finished" podID="266d8ee2-7184-4f2f-b1dd-cdea44709d9f" containerID="f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875" exitCode=0
Jul 15 21:11:42.718859 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.718457    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b868d9f9-5dwrs"
Jul 15 21:11:42.718859 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.718496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b868d9f9-5dwrs" event={"ID":"266d8ee2-7184-4f2f-b1dd-cdea44709d9f","Type":"ContainerDied","Data":"f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875"}
Jul 15 21:11:42.718859 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.718554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b868d9f9-5dwrs" event={"ID":"266d8ee2-7184-4f2f-b1dd-cdea44709d9f","Type":"ContainerDied","Data":"005ab352412c54cd748050dad567da6cccb77d051fc326ea3cd76fcf7b79faec"}
Jul 15 21:11:42.718859 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.718576    2580 scope.go:117] "RemoveContainer" containerID="f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875"
Jul 15 21:11:42.727164 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.727148    2580 scope.go:117] "RemoveContainer" containerID="f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875"
Jul 15 21:11:42.727401 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:11:42.727383    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875\": container with ID starting with f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875 not found: ID does not exist" containerID="f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875"
Jul 15 21:11:42.727471 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.727410    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875"} err="failed to get container status \"f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875\": rpc error: code = NotFound desc = could not find container \"f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875\": container with ID starting with f03a342bf58398bce787bfb9b1bdc9f53fb32c4325a168739a75e639254ea875 not found: ID does not exist"
Jul 15 21:11:42.747135 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.747112    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b868d9f9-5dwrs"]
Jul 15 21:11:42.748606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:42.748583    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b868d9f9-5dwrs"]
Jul 15 21:11:43.134075 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:11:43.134046    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="266d8ee2-7184-4f2f-b1dd-cdea44709d9f" path="/var/lib/kubelet/pods/266d8ee2-7184-4f2f-b1dd-cdea44709d9f/volumes"
Jul 15 21:12:02.329604 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.329567    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"]
Jul 15 21:12:02.330105 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.329918    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="266d8ee2-7184-4f2f-b1dd-cdea44709d9f" containerName="manager"
Jul 15 21:12:02.330105 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.329931    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="266d8ee2-7184-4f2f-b1dd-cdea44709d9f" containerName="manager"
Jul 15 21:12:02.330105 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.330035    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="266d8ee2-7184-4f2f-b1dd-cdea44709d9f" containerName="manager"
Jul 15 21:12:02.334622 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.334602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.337333 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.337311    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 21:12:02.337521 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.337508    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 21:12:02.337606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.337590    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 21:12:02.344332 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.344308    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"]
Jul 15 21:12:02.451058 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.451021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m7qn\" (UniqueName: \"kubernetes.io/projected/82da60cd-987f-4823-80ff-8460151853ca-kube-api-access-9m7qn\") pod \"maas-api-5fd78dcc94-ltqxg\" (UID: \"82da60cd-987f-4823-80ff-8460151853ca\") " pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.451058 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.451060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82da60cd-987f-4823-80ff-8460151853ca-maas-api-tls\") pod \"maas-api-5fd78dcc94-ltqxg\" (UID: \"82da60cd-987f-4823-80ff-8460151853ca\") " pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.451261 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.451173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82da60cd-987f-4823-80ff-8460151853ca-openshift-service-ca\") pod \"maas-api-5fd78dcc94-ltqxg\" (UID: \"82da60cd-987f-4823-80ff-8460151853ca\") " pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.551649 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.551609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82da60cd-987f-4823-80ff-8460151853ca-openshift-service-ca\") pod \"maas-api-5fd78dcc94-ltqxg\" (UID: \"82da60cd-987f-4823-80ff-8460151853ca\") " pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.551838 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.551678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m7qn\" (UniqueName: \"kubernetes.io/projected/82da60cd-987f-4823-80ff-8460151853ca-kube-api-access-9m7qn\") pod \"maas-api-5fd78dcc94-ltqxg\" (UID: \"82da60cd-987f-4823-80ff-8460151853ca\") " pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.551838 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.551699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82da60cd-987f-4823-80ff-8460151853ca-maas-api-tls\") pod \"maas-api-5fd78dcc94-ltqxg\" (UID: \"82da60cd-987f-4823-80ff-8460151853ca\") " pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.552427 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.552399    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82da60cd-987f-4823-80ff-8460151853ca-openshift-service-ca\") pod \"maas-api-5fd78dcc94-ltqxg\" (UID: \"82da60cd-987f-4823-80ff-8460151853ca\") " pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.554835 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.554734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82da60cd-987f-4823-80ff-8460151853ca-maas-api-tls\") pod \"maas-api-5fd78dcc94-ltqxg\" (UID: \"82da60cd-987f-4823-80ff-8460151853ca\") " pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.560198 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.560163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m7qn\" (UniqueName: \"kubernetes.io/projected/82da60cd-987f-4823-80ff-8460151853ca-kube-api-access-9m7qn\") pod \"maas-api-5fd78dcc94-ltqxg\" (UID: \"82da60cd-987f-4823-80ff-8460151853ca\") " pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.645138 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.645058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:02.788462 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.788440    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"]
Jul 15 21:12:02.791212 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:12:02.791185    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82da60cd_987f_4823_80ff_8460151853ca.slice/crio-339884d4fba0af44c5d0505b47039b1c5b80a2ca23322fde91d088336371d623 WatchSource:0}: Error finding container 339884d4fba0af44c5d0505b47039b1c5b80a2ca23322fde91d088336371d623: Status 404 returned error can't find the container with id 339884d4fba0af44c5d0505b47039b1c5b80a2ca23322fde91d088336371d623
Jul 15 21:12:02.815095 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.815067    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg" event={"ID":"82da60cd-987f-4823-80ff-8460151853ca","Type":"ContainerStarted","Data":"339884d4fba0af44c5d0505b47039b1c5b80a2ca23322fde91d088336371d623"}
Jul 15 21:12:02.849306 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.849273    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"]
Jul 15 21:12:02.855935 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.854758    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:12:02.857638 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.857608    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 21:12:02.857760 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.857740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gvtp5\""
Jul 15 21:12:02.864016 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.863989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"]
Jul 15 21:12:02.868322 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.868298    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"]
Jul 15 21:12:02.870676 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.870654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:02.881663 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.881634    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"]
Jul 15 21:12:02.958091 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.957993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4psm\" (UniqueName: \"kubernetes.io/projected/16d252fd-b5c2-4179-9bdd-cde357c9d355-kube-api-access-s4psm\") pod \"payload-processing-f9b8d9b9b-bpf2x\" (UID: \"16d252fd-b5c2-4179-9bdd-cde357c9d355\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:02.958091 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.958080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e3752e-0960-4f15-be87-6e1ee6c81472-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-4lwtr\" (UID: \"e3e3752e-0960-4f15-be87-6e1ee6c81472\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:12:02.958320 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.958154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d252fd-b5c2-4179-9bdd-cde357c9d355-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-bpf2x\" (UID: \"16d252fd-b5c2-4179-9bdd-cde357c9d355\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:02.958320 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:02.958198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dcx4\" (UniqueName: \"kubernetes.io/projected/e3e3752e-0960-4f15-be87-6e1ee6c81472-kube-api-access-8dcx4\") pod \"payload-pre-processing-74584bb9f8-4lwtr\" (UID: \"e3e3752e-0960-4f15-be87-6e1ee6c81472\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:12:03.059135 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.059102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dcx4\" (UniqueName: \"kubernetes.io/projected/e3e3752e-0960-4f15-be87-6e1ee6c81472-kube-api-access-8dcx4\") pod \"payload-pre-processing-74584bb9f8-4lwtr\" (UID: \"e3e3752e-0960-4f15-be87-6e1ee6c81472\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:12:03.059326 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.059173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4psm\" (UniqueName: \"kubernetes.io/projected/16d252fd-b5c2-4179-9bdd-cde357c9d355-kube-api-access-s4psm\") pod \"payload-processing-f9b8d9b9b-bpf2x\" (UID: \"16d252fd-b5c2-4179-9bdd-cde357c9d355\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:03.059326 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.059216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e3752e-0960-4f15-be87-6e1ee6c81472-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-4lwtr\" (UID: \"e3e3752e-0960-4f15-be87-6e1ee6c81472\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:12:03.059326 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.059270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d252fd-b5c2-4179-9bdd-cde357c9d355-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-bpf2x\" (UID: \"16d252fd-b5c2-4179-9bdd-cde357c9d355\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:03.060010 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.059986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e3752e-0960-4f15-be87-6e1ee6c81472-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-4lwtr\" (UID: \"e3e3752e-0960-4f15-be87-6e1ee6c81472\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:12:03.060063 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.060005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d252fd-b5c2-4179-9bdd-cde357c9d355-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-bpf2x\" (UID: \"16d252fd-b5c2-4179-9bdd-cde357c9d355\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:03.068304 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.068283    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dcx4\" (UniqueName: \"kubernetes.io/projected/e3e3752e-0960-4f15-be87-6e1ee6c81472-kube-api-access-8dcx4\") pod \"payload-pre-processing-74584bb9f8-4lwtr\" (UID: \"e3e3752e-0960-4f15-be87-6e1ee6c81472\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:12:03.068407 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.068282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4psm\" (UniqueName: \"kubernetes.io/projected/16d252fd-b5c2-4179-9bdd-cde357c9d355-kube-api-access-s4psm\") pod \"payload-processing-f9b8d9b9b-bpf2x\" (UID: \"16d252fd-b5c2-4179-9bdd-cde357c9d355\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:03.168831 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.168794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:12:03.181185 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.181150    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:03.310608 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.310574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"]
Jul 15 21:12:03.312480 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:12:03.312452    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e3752e_0960_4f15_be87_6e1ee6c81472.slice/crio-2b3a30406d50f3e126e16266ce00676eb051e2bff432cc4b779eee034ca3e0e3 WatchSource:0}: Error finding container 2b3a30406d50f3e126e16266ce00676eb051e2bff432cc4b779eee034ca3e0e3: Status 404 returned error can't find the container with id 2b3a30406d50f3e126e16266ce00676eb051e2bff432cc4b779eee034ca3e0e3
Jul 15 21:12:03.337451 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.337429    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"]
Jul 15 21:12:03.339451 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:12:03.339419    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d252fd_b5c2_4179_9bdd_cde357c9d355.slice/crio-92afac9c4a7064d41ed54f0f81dd3425c6492eb30a13356012fbce0b177834e5 WatchSource:0}: Error finding container 92afac9c4a7064d41ed54f0f81dd3425c6492eb30a13356012fbce0b177834e5: Status 404 returned error can't find the container with id 92afac9c4a7064d41ed54f0f81dd3425c6492eb30a13356012fbce0b177834e5
Jul 15 21:12:03.820943 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.820910    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x" event={"ID":"16d252fd-b5c2-4179-9bdd-cde357c9d355","Type":"ContainerStarted","Data":"92afac9c4a7064d41ed54f0f81dd3425c6492eb30a13356012fbce0b177834e5"}
Jul 15 21:12:03.822012 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:03.821989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr" event={"ID":"e3e3752e-0960-4f15-be87-6e1ee6c81472","Type":"ContainerStarted","Data":"2b3a30406d50f3e126e16266ce00676eb051e2bff432cc4b779eee034ca3e0e3"}
Jul 15 21:12:05.834139 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:05.834101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x" event={"ID":"16d252fd-b5c2-4179-9bdd-cde357c9d355","Type":"ContainerStarted","Data":"69ebcd71cbdc38c45be427e3138ee22b57d11163980eac826b8a7828fc2f3c32"}
Jul 15 21:12:05.834585 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:05.834180    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:05.835599 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:05.835573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg" event={"ID":"82da60cd-987f-4823-80ff-8460151853ca","Type":"ContainerStarted","Data":"e1c4c9958565e9c6af9aa35985973dfcd4a9de7e3e2f4e937395d60e96848a28"}
Jul 15 21:12:05.835660 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:05.835633    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:05.837009 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:05.836979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr" event={"ID":"e3e3752e-0960-4f15-be87-6e1ee6c81472","Type":"ContainerStarted","Data":"c341a43a47b75c4f8e35103c3fb809ba93a4092ca82ddca877fe4b75cf529bbd"}
Jul 15 21:12:05.837143 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:05.837112    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:12:05.851267 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:05.851216    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x" podStartSLOduration=1.883345994 podStartE2EDuration="3.851198378s" totalImagesPullingTime="1.967852384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:12:05.850496825 +0000 UTC m=+595.207415423" watchObservedRunningTime="2026-07-15 21:12:05.851198378 +0000 UTC m=+595.208116976"
Jul 15 21:12:05.871886 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:05.871832    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr" podStartSLOduration=1.8800350959999999 podStartE2EDuration="3.871811434s" totalImagesPullingTime="1.991776338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:12:05.86899044 +0000 UTC m=+595.225909029" watchObservedRunningTime="2026-07-15 21:12:05.871811434 +0000 UTC m=+595.228730013"
Jul 15 21:12:05.887460 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:05.887410    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg" podStartSLOduration=1.373071048 podStartE2EDuration="3.887389296s" totalImagesPullingTime="2.514318248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:12:05.886831049 +0000 UTC m=+595.243749648" watchObservedRunningTime="2026-07-15 21:12:05.887389296 +0000 UTC m=+595.244307895"
Jul 15 21:12:11.033746 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:11.033715    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:12:11.034486 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:11.034467    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:12:11.037165 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:11.037142    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:12:11.038290 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:11.038261    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:12:11.847766 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:11.847733    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5fd78dcc94-ltqxg"
Jul 15 21:12:16.844317 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:16.844281    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-bpf2x"
Jul 15 21:12:16.844773 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:12:16.844498    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-4lwtr"
Jul 15 21:13:01.297001 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.296969    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"]
Jul 15 21:13:01.300945 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.300925    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.303215 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.303197    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 21:13:01.303413 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.303395    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 21:13:01.303863 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.303847    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mxvc9\""
Jul 15 21:13:01.303975 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.303888    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 15 21:13:01.311035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.311012    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"]
Jul 15 21:13:01.460273 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.460235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zd5l\" (UniqueName: \"kubernetes.io/projected/7421969e-1676-4071-9336-0c601e1eb2fa-kube-api-access-2zd5l\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.460445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.460289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7421969e-1676-4071-9336-0c601e1eb2fa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.460445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.460348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.460445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.460420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.460565 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.460445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.460607 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.460557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.561693 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.561613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.561693 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.561658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.561693 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.561674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.561981 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.561722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.561981 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.561750    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zd5l\" (UniqueName: \"kubernetes.io/projected/7421969e-1676-4071-9336-0c601e1eb2fa-kube-api-access-2zd5l\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.561981 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.561774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7421969e-1676-4071-9336-0c601e1eb2fa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.562149 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.562052    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.562192 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.562154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.562226 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.562191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.563926 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.563902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7421969e-1676-4071-9336-0c601e1eb2fa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.564255 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.564237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7421969e-1676-4071-9336-0c601e1eb2fa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.574845 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.574823    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zd5l\" (UniqueName: \"kubernetes.io/projected/7421969e-1676-4071-9336-0c601e1eb2fa-kube-api-access-2zd5l\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vlf8x\" (UID: \"7421969e-1676-4071-9336-0c601e1eb2fa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.612710 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.612686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:01.748001 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.747973    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"]
Jul 15 21:13:01.749991 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:13:01.749944    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7421969e_1676_4071_9336_0c601e1eb2fa.slice/crio-eacc59d4c618595cc0f7c7c29ad7c6dca8ada67b749558266369209cc71056c1 WatchSource:0}: Error finding container eacc59d4c618595cc0f7c7c29ad7c6dca8ada67b749558266369209cc71056c1: Status 404 returned error can't find the container with id eacc59d4c618595cc0f7c7c29ad7c6dca8ada67b749558266369209cc71056c1
Jul 15 21:13:01.751289 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:01.751271    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:13:02.117079 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:02.116996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x" event={"ID":"7421969e-1676-4071-9336-0c601e1eb2fa","Type":"ContainerStarted","Data":"eacc59d4c618595cc0f7c7c29ad7c6dca8ada67b749558266369209cc71056c1"}
Jul 15 21:13:03.123344 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:03.123305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x" event={"ID":"7421969e-1676-4071-9336-0c601e1eb2fa","Type":"ContainerStarted","Data":"ea94d2b2e9f00308d4a9fb709ded9260f83498775ff03d97aad0b844a21f6388"}
Jul 15 21:13:03.145569 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:03.145514    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x" podStartSLOduration=1.123470176 podStartE2EDuration="2.145495266s" totalImagesPullingTime="1.02202509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:03.144616308 +0000 UTC m=+652.501534905" watchObservedRunningTime="2026-07-15 21:13:03.145495266 +0000 UTC m=+652.502413862"
Jul 15 21:13:11.613038 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:11.612997    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:11.613460 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:11.613084    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:11.625689 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:11.625665    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:12.176228 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:12.176199    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vlf8x"
Jul 15 21:13:23.486481 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.486442    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"]
Jul 15 21:13:23.491054 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.491030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.493532 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.493511    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 15 21:13:23.497806 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.497780    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"]
Jul 15 21:13:23.556643 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.556606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb403a4-2722-48e4-afb2-52f9e2e75480-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.556822 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.556674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67nz9\" (UniqueName: \"kubernetes.io/projected/ffb403a4-2722-48e4-afb2-52f9e2e75480-kube-api-access-67nz9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.556822 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.556706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.556822 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.556727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.556822 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.556809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.557038 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.556852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.658354 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.658310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67nz9\" (UniqueName: \"kubernetes.io/projected/ffb403a4-2722-48e4-afb2-52f9e2e75480-kube-api-access-67nz9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.658556 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.658368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.658556 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.658400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.658556 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.658445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.658556 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.658495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.658556 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.658548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb403a4-2722-48e4-afb2-52f9e2e75480-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.658936 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.658915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.659049 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.658948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.659049 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.659001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.660782 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.660763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb403a4-2722-48e4-afb2-52f9e2e75480-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.661141 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.661121    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb403a4-2722-48e4-afb2-52f9e2e75480-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.666685 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.666665    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67nz9\" (UniqueName: \"kubernetes.io/projected/ffb403a4-2722-48e4-afb2-52f9e2e75480-kube-api-access-67nz9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xg8tp\" (UID: \"ffb403a4-2722-48e4-afb2-52f9e2e75480\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.802785 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.802731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:23.940458 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:23.940424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"]
Jul 15 21:13:23.944403 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:13:23.944368    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb403a4_2722_48e4_afb2_52f9e2e75480.slice/crio-2d1e58b8cd5a830a5049cd323f8085c935ae2dd73c529cabc51d7c401ff4c6ab WatchSource:0}: Error finding container 2d1e58b8cd5a830a5049cd323f8085c935ae2dd73c529cabc51d7c401ff4c6ab: Status 404 returned error can't find the container with id 2d1e58b8cd5a830a5049cd323f8085c935ae2dd73c529cabc51d7c401ff4c6ab
Jul 15 21:13:24.217872 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:24.217840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp" event={"ID":"ffb403a4-2722-48e4-afb2-52f9e2e75480","Type":"ContainerStarted","Data":"c8c68e9f7cf018a8f333740f0220f10c691e2d879511a6ed369e4f051d99bf72"}
Jul 15 21:13:24.218084 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:24.217898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp" event={"ID":"ffb403a4-2722-48e4-afb2-52f9e2e75480","Type":"ContainerStarted","Data":"2d1e58b8cd5a830a5049cd323f8085c935ae2dd73c529cabc51d7c401ff4c6ab"}
Jul 15 21:13:24.235967 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:24.235909    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp" podStartSLOduration=1.050485026 podStartE2EDuration="1.235890312s" totalImagesPullingTime="185.405286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:13:24.234218019 +0000 UTC m=+673.591136631" watchObservedRunningTime="2026-07-15 21:13:24.235890312 +0000 UTC m=+673.592808908"
Jul 15 21:13:33.803820 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:33.803767    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:33.804255 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:33.803847    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:33.816547 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:33.816521    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:13:34.275483 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:13:34.275400    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xg8tp"
Jul 15 21:15:00.154136 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:00.154095    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv"]
Jul 15 21:15:00.157631 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:00.157612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv"
Jul 15 21:15:00.160201 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:00.160183    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:15:00.167081 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:00.167054    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv"]
Jul 15 21:15:00.168689 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:00.168665    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv"
Jul 15 21:15:00.316139 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:00.316109    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv"]
Jul 15 21:15:00.318653 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:15:00.318628    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b899e9d_5d54_40b1_b104_dbd74ba0166b.slice/crio-a287c8c4717b096349c22b416e91067b137d27e9dce7968f5e7e41e237e4e8f3 WatchSource:0}: Error finding container a287c8c4717b096349c22b416e91067b137d27e9dce7968f5e7e41e237e4e8f3: Status 404 returned error can't find the container with id a287c8c4717b096349c22b416e91067b137d27e9dce7968f5e7e41e237e4e8f3
Jul 15 21:15:00.668375 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:00.668337    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv" event={"ID":"7b899e9d-5d54-40b1-b104-dbd74ba0166b","Type":"ContainerStarted","Data":"a287c8c4717b096349c22b416e91067b137d27e9dce7968f5e7e41e237e4e8f3"}
Jul 15 21:15:02.679621 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:02.679586    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv" event={"ID":"7b899e9d-5d54-40b1-b104-dbd74ba0166b","Type":"ContainerStarted","Data":"b53b7ac4c5ba4ad7f476555e1b14cb9ecbdde9e07bce0508849825775167b370"}
Jul 15 21:15:02.695382 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:02.695334    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv" podStartSLOduration=0.517189338 podStartE2EDuration="2.695316992s" totalImagesPullingTime="2.178127654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:15:02.694613889 +0000 UTC m=+772.051532487" watchObservedRunningTime="2026-07-15 21:15:02.695316992 +0000 UTC m=+772.052235589"
Jul 15 21:15:03.686256 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:03.686219    2580 generic.go:356] "Generic (PLEG): container finished" podID="7b899e9d-5d54-40b1-b104-dbd74ba0166b" containerID="b53b7ac4c5ba4ad7f476555e1b14cb9ecbdde9e07bce0508849825775167b370" exitCode=0
Jul 15 21:15:03.686664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:03.686284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv" event={"ID":"7b899e9d-5d54-40b1-b104-dbd74ba0166b","Type":"ContainerDied","Data":"b53b7ac4c5ba4ad7f476555e1b14cb9ecbdde9e07bce0508849825775167b370"}
Jul 15 21:15:04.833758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:04.833734    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv"
Jul 15 21:15:05.697280 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:05.697242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv" event={"ID":"7b899e9d-5d54-40b1-b104-dbd74ba0166b","Type":"ContainerDied","Data":"a287c8c4717b096349c22b416e91067b137d27e9dce7968f5e7e41e237e4e8f3"}
Jul 15 21:15:05.697280 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:05.697262    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-mtbvv"
Jul 15 21:15:05.697648 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:05.697292    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a287c8c4717b096349c22b416e91067b137d27e9dce7968f5e7e41e237e4e8f3"
Jul 15 21:15:56.077839 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.077799    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b45b4f795-k985s"]
Jul 15 21:15:56.078348 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.078029    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b45b4f795-k985s" podUID="c1eda09d-f843-405f-bfb3-ee9356ab5246" containerName="manager" containerID="cri-o://51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7" gracePeriod=30
Jul 15 21:15:56.327202 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.327171    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:15:56.401076 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.400989    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9flk\" (UniqueName: \"kubernetes.io/projected/c1eda09d-f843-405f-bfb3-ee9356ab5246-kube-api-access-g9flk\") pod \"c1eda09d-f843-405f-bfb3-ee9356ab5246\" (UID: \"c1eda09d-f843-405f-bfb3-ee9356ab5246\") "
Jul 15 21:15:56.401076 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.401065    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1eda09d-f843-405f-bfb3-ee9356ab5246-webhook-cert\") pod \"c1eda09d-f843-405f-bfb3-ee9356ab5246\" (UID: \"c1eda09d-f843-405f-bfb3-ee9356ab5246\") "
Jul 15 21:15:56.403166 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.403137    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1eda09d-f843-405f-bfb3-ee9356ab5246-kube-api-access-g9flk" (OuterVolumeSpecName: "kube-api-access-g9flk") pod "c1eda09d-f843-405f-bfb3-ee9356ab5246" (UID: "c1eda09d-f843-405f-bfb3-ee9356ab5246"). InnerVolumeSpecName "kube-api-access-g9flk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:15:56.403268 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.403171    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1eda09d-f843-405f-bfb3-ee9356ab5246-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c1eda09d-f843-405f-bfb3-ee9356ab5246" (UID: "c1eda09d-f843-405f-bfb3-ee9356ab5246"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:15:56.502519 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.502476    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9flk\" (UniqueName: \"kubernetes.io/projected/c1eda09d-f843-405f-bfb3-ee9356ab5246-kube-api-access-g9flk\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:15:56.502519 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.502505    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1eda09d-f843-405f-bfb3-ee9356ab5246-webhook-cert\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:15:56.947450 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.947417    2580 generic.go:356] "Generic (PLEG): container finished" podID="c1eda09d-f843-405f-bfb3-ee9356ab5246" containerID="51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7" exitCode=0
Jul 15 21:15:56.947627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.947460    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b45b4f795-k985s"
Jul 15 21:15:56.947627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.947508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b45b4f795-k985s" event={"ID":"c1eda09d-f843-405f-bfb3-ee9356ab5246","Type":"ContainerDied","Data":"51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7"}
Jul 15 21:15:56.947627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.947561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b45b4f795-k985s" event={"ID":"c1eda09d-f843-405f-bfb3-ee9356ab5246","Type":"ContainerDied","Data":"f094fcec8fe6792fab61415c13f5913cf3fdf514d2fbc7100f81427f5b360146"}
Jul 15 21:15:56.947627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.947584    2580 scope.go:117] "RemoveContainer" containerID="51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7"
Jul 15 21:15:56.958436 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.958417    2580 scope.go:117] "RemoveContainer" containerID="51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7"
Jul 15 21:15:56.958730 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:15:56.958710    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7\": container with ID starting with 51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7 not found: ID does not exist" containerID="51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7"
Jul 15 21:15:56.958779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.958739    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7"} err="failed to get container status \"51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7\": rpc error: code = NotFound desc = could not find container \"51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7\": container with ID starting with 51c801247c972aa3634d9e0ee39e006a412ec46513481fa9311f9cb08267eca7 not found: ID does not exist"
Jul 15 21:15:56.980486 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.980453    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b45b4f795-k985s"]
Jul 15 21:15:56.982161 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:56.982136    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b45b4f795-k985s"]
Jul 15 21:15:57.134350 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.134315    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1eda09d-f843-405f-bfb3-ee9356ab5246" path="/var/lib/kubelet/pods/c1eda09d-f843-405f-bfb3-ee9356ab5246/volumes"
Jul 15 21:15:57.777435 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.777404    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b45b4f795-9kmmd"]
Jul 15 21:15:57.777776 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.777765    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b899e9d-5d54-40b1-b104-dbd74ba0166b" containerName="cleanup"
Jul 15 21:15:57.777826 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.777778    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b899e9d-5d54-40b1-b104-dbd74ba0166b" containerName="cleanup"
Jul 15 21:15:57.777826 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.777788    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1eda09d-f843-405f-bfb3-ee9356ab5246" containerName="manager"
Jul 15 21:15:57.777826 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.777793    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1eda09d-f843-405f-bfb3-ee9356ab5246" containerName="manager"
Jul 15 21:15:57.777929 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.777855    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b899e9d-5d54-40b1-b104-dbd74ba0166b" containerName="cleanup"
Jul 15 21:15:57.777929 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.777864    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1eda09d-f843-405f-bfb3-ee9356ab5246" containerName="manager"
Jul 15 21:15:57.782616 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.782594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:15:57.785762 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.785735    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 21:15:57.785894 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.785787    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7f6jx\""
Jul 15 21:15:57.785894 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.785854    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 21:15:57.791706 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.791682    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b45b4f795-9kmmd"]
Jul 15 21:15:57.815506 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.815469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sprcg\" (UniqueName: \"kubernetes.io/projected/8291a711-84ae-42ff-ba75-02528d6525de-kube-api-access-sprcg\") pod \"maas-controller-6b45b4f795-9kmmd\" (UID: \"8291a711-84ae-42ff-ba75-02528d6525de\") " pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:15:57.815684 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.815543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8291a711-84ae-42ff-ba75-02528d6525de-webhook-cert\") pod \"maas-controller-6b45b4f795-9kmmd\" (UID: \"8291a711-84ae-42ff-ba75-02528d6525de\") " pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:15:57.916362 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.916321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sprcg\" (UniqueName: \"kubernetes.io/projected/8291a711-84ae-42ff-ba75-02528d6525de-kube-api-access-sprcg\") pod \"maas-controller-6b45b4f795-9kmmd\" (UID: \"8291a711-84ae-42ff-ba75-02528d6525de\") " pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:15:57.916540 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.916396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8291a711-84ae-42ff-ba75-02528d6525de-webhook-cert\") pod \"maas-controller-6b45b4f795-9kmmd\" (UID: \"8291a711-84ae-42ff-ba75-02528d6525de\") " pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:15:57.918883 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.918854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8291a711-84ae-42ff-ba75-02528d6525de-webhook-cert\") pod \"maas-controller-6b45b4f795-9kmmd\" (UID: \"8291a711-84ae-42ff-ba75-02528d6525de\") " pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:15:57.924687 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:57.924668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sprcg\" (UniqueName: \"kubernetes.io/projected/8291a711-84ae-42ff-ba75-02528d6525de-kube-api-access-sprcg\") pod \"maas-controller-6b45b4f795-9kmmd\" (UID: \"8291a711-84ae-42ff-ba75-02528d6525de\") " pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:15:58.093860 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:58.093760    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:15:58.228096 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:58.228066    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b45b4f795-9kmmd"]
Jul 15 21:15:58.231935 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:15:58.231907    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8291a711_84ae_42ff_ba75_02528d6525de.slice/crio-a100a8720df3b929421a8574a54374a7cffcbeeb6f0d0618c9507ef93b531f16 WatchSource:0}: Error finding container a100a8720df3b929421a8574a54374a7cffcbeeb6f0d0618c9507ef93b531f16: Status 404 returned error can't find the container with id a100a8720df3b929421a8574a54374a7cffcbeeb6f0d0618c9507ef93b531f16
Jul 15 21:15:58.961177 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:58.961079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b45b4f795-9kmmd" event={"ID":"8291a711-84ae-42ff-ba75-02528d6525de","Type":"ContainerStarted","Data":"4a975b605f3e746384853e48388fce8377d54a1f463affac90ee310fe081f470"}
Jul 15 21:15:58.961177 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:58.961146    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:15:58.961177 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:58.961170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b45b4f795-9kmmd" event={"ID":"8291a711-84ae-42ff-ba75-02528d6525de","Type":"ContainerStarted","Data":"a100a8720df3b929421a8574a54374a7cffcbeeb6f0d0618c9507ef93b531f16"}
Jul 15 21:15:58.981672 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:15:58.981611    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b45b4f795-9kmmd" podStartSLOduration=1.5661427300000001 podStartE2EDuration="1.98159198s" totalImagesPullingTime="415.44925ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:15:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:15:58.981020136 +0000 UTC m=+828.337938732" watchObservedRunningTime="2026-07-15 21:15:58.98159198 +0000 UTC m=+828.338510576"
Jul 15 21:16:09.968688 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:16:09.968655    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:17:11.066405 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:17:11.066328    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:17:11.066918 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:17:11.066796    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:17:11.070381 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:17:11.070360    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:17:11.070991 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:17:11.070953    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:22:11.099707 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:22:11.099672    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:22:11.103347 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:22:11.103307    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:22:11.103648 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:22:11.103629    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:22:11.107243 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:22:11.107220    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:25:03.585298 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:25:03.585265    2580 generic.go:356] "Generic (PLEG): container finished" podID="8291a711-84ae-42ff-ba75-02528d6525de" containerID="4a975b605f3e746384853e48388fce8377d54a1f463affac90ee310fe081f470" exitCode=1
Jul 15 21:25:03.585717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:25:03.585343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b45b4f795-9kmmd" event={"ID":"8291a711-84ae-42ff-ba75-02528d6525de","Type":"ContainerDied","Data":"4a975b605f3e746384853e48388fce8377d54a1f463affac90ee310fe081f470"}
Jul 15 21:25:03.585717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:25:03.585674    2580 scope.go:117] "RemoveContainer" containerID="4a975b605f3e746384853e48388fce8377d54a1f463affac90ee310fe081f470"
Jul 15 21:25:03.585988 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:25:03.585976    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:25:04.377556 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:25:04.377537    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 21:25:04.591731 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:25:04.591693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b45b4f795-9kmmd" event={"ID":"8291a711-84ae-42ff-ba75-02528d6525de","Type":"ContainerStarted","Data":"0a980c27b5c10033ea7ac46aed28ab4ec2937b46bae4ab8003fe13ea91fbb79a"}
Jul 15 21:25:04.592152 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:25:04.591925    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:25:08.439008 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:25:08.438936    2580 controller.go:195] "Failed to update lease" err="Put \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-130-29.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 15 21:25:15.597926 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:25:15.597876    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-6b45b4f795-9kmmd" podUID="8291a711-84ae-42ff-ba75-02528d6525de" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.40:8081/readyz\": dial tcp 10.134.0.40:8081: connect: connection refused"
Jul 15 21:25:17.574377 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:25:17.574336    2580 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-130-29.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again"
Jul 15 21:25:25.597505 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:25:25.597463    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b45b4f795-9kmmd"
Jul 15 21:25:38.957287 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:25:38.957234    2580 controller.go:195] "Failed to update lease" err="Put \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-130-29.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 15 21:25:48.957686 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:25:48.957634    2580 controller.go:195] "Failed to update lease" err="Put \"https://a9c0189c76ffa414bbe64782f22ffef9-e324037f91a3c3d1.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-130-29.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 15 21:25:49.306814 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:25:49.306777    2580 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-130-29.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again"
Jul 15 21:27:11.148231 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:27:11.148197    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:27:11.151974 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:27:11.151929    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:27:11.153837 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:27:11.153813    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:27:11.157479 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:27:11.157462    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:29:45.742724 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.742689    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"]
Jul 15 21:29:45.746191 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.746168    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:45.748747 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.748714    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8cee6bf5-serving-cert\""
Jul 15 21:29:45.748747 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.748721    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hxcvg\""
Jul 15 21:29:45.771534 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.771502    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"]
Jul 15 21:29:45.833386 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.833349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86zmk\" (UniqueName: \"kubernetes.io/projected/c9baf188-3b96-472b-ba9e-0dc4cce534ac-kube-api-access-86zmk\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:45.833556 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.833391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9baf188-3b96-472b-ba9e-0dc4cce534ac-openshift-service-ca\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:45.833556 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.833410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9baf188-3b96-472b-ba9e-0dc4cce534ac-maas-api-tls\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:45.934241 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.934207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zmk\" (UniqueName: \"kubernetes.io/projected/c9baf188-3b96-472b-ba9e-0dc4cce534ac-kube-api-access-86zmk\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:45.934241 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.934245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9baf188-3b96-472b-ba9e-0dc4cce534ac-openshift-service-ca\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:45.934501 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.934263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9baf188-3b96-472b-ba9e-0dc4cce534ac-maas-api-tls\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:45.934501 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:29:45.934388    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-serving-cert: secret "maas-api-e2e-ait-8cee6bf5-serving-cert" not found
Jul 15 21:29:45.934501 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:29:45.934444    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9baf188-3b96-472b-ba9e-0dc4cce534ac-maas-api-tls podName:c9baf188-3b96-472b-ba9e-0dc4cce534ac nodeName:}" failed. No retries permitted until 2026-07-15 21:29:46.434430002 +0000 UTC m=+1655.791348576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9baf188-3b96-472b-ba9e-0dc4cce534ac-maas-api-tls") pod "maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f" (UID: "c9baf188-3b96-472b-ba9e-0dc4cce534ac") : secret "maas-api-e2e-ait-8cee6bf5-serving-cert" not found
Jul 15 21:29:45.934954 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.934926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9baf188-3b96-472b-ba9e-0dc4cce534ac-openshift-service-ca\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:45.946654 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:45.946627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86zmk\" (UniqueName: \"kubernetes.io/projected/c9baf188-3b96-472b-ba9e-0dc4cce534ac-kube-api-access-86zmk\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:46.438543 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:46.438488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9baf188-3b96-472b-ba9e-0dc4cce534ac-maas-api-tls\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:46.440939 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:46.440917    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9baf188-3b96-472b-ba9e-0dc4cce534ac-maas-api-tls\") pod \"maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:46.656749 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:46.656712    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:46.801250 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:46.801219    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"]
Jul 15 21:29:46.803606 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:29:46.803578    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9baf188_3b96_472b_ba9e_0dc4cce534ac.slice/crio-c1def50d58a4471a35f51b18129b4a556f0d1188b6eefd5964280a1232e73694 WatchSource:0}: Error finding container c1def50d58a4471a35f51b18129b4a556f0d1188b6eefd5964280a1232e73694: Status 404 returned error can't find the container with id c1def50d58a4471a35f51b18129b4a556f0d1188b6eefd5964280a1232e73694
Jul 15 21:29:46.934900 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:46.934815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f" event={"ID":"c9baf188-3b96-472b-ba9e-0dc4cce534ac","Type":"ContainerStarted","Data":"c1def50d58a4471a35f51b18129b4a556f0d1188b6eefd5964280a1232e73694"}
Jul 15 21:29:47.940444 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:47.940351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f" event={"ID":"c9baf188-3b96-472b-ba9e-0dc4cce534ac","Type":"ContainerStarted","Data":"5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448"}
Jul 15 21:29:47.940852 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:47.940489    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:47.960079 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:47.960026    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f" podStartSLOduration=2.200603312 podStartE2EDuration="2.960006785s" totalImagesPullingTime="759.403473ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:29:47.959314764 +0000 UTC m=+1657.316233360" watchObservedRunningTime="2026-07-15 21:29:47.960006785 +0000 UTC m=+1657.316925382"
Jul 15 21:29:50.532454 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:50.532423    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"]
Jul 15 21:29:50.535996 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:50.535979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"
Jul 15 21:29:50.539029 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:50.539009    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:29:50.555669 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:50.555646    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"]
Jul 15 21:29:50.679125 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:50.679090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/998cb43f-088d-4c4f-bfb8-275b643ba1f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf\" (UID: \"998cb43f-088d-4c4f-bfb8-275b643ba1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"
Jul 15 21:29:50.780396 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:50.780348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/998cb43f-088d-4c4f-bfb8-275b643ba1f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf\" (UID: \"998cb43f-088d-4c4f-bfb8-275b643ba1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"
Jul 15 21:29:50.781008 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:50.780986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/998cb43f-088d-4c4f-bfb8-275b643ba1f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf\" (UID: \"998cb43f-088d-4c4f-bfb8-275b643ba1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"
Jul 15 21:29:50.847103 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:50.847017    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"
Jul 15 21:29:50.995390 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:50.995357    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"]
Jul 15 21:29:50.998389 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:29:50.998358    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998cb43f_088d_4c4f_bfb8_275b643ba1f7.slice/crio-4d1201b6665cde6fe2235088126108a5cf5b5ef7639cdcef4d9f4fdb8c1b03df WatchSource:0}: Error finding container 4d1201b6665cde6fe2235088126108a5cf5b5ef7639cdcef4d9f4fdb8c1b03df: Status 404 returned error can't find the container with id 4d1201b6665cde6fe2235088126108a5cf5b5ef7639cdcef4d9f4fdb8c1b03df
Jul 15 21:29:51.964257 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:51.964166    2580 generic.go:356] "Generic (PLEG): container finished" podID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerID="9adc8c00a58db1e4087bd685d35dabf2f68cf267eb6f8f2c3420c13cc89fa601" exitCode=7
Jul 15 21:29:51.964257 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:51.964246    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf" event={"ID":"998cb43f-088d-4c4f-bfb8-275b643ba1f7","Type":"ContainerDied","Data":"9adc8c00a58db1e4087bd685d35dabf2f68cf267eb6f8f2c3420c13cc89fa601"}
Jul 15 21:29:51.964627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:51.964301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf" event={"ID":"998cb43f-088d-4c4f-bfb8-275b643ba1f7","Type":"ContainerStarted","Data":"4d1201b6665cde6fe2235088126108a5cf5b5ef7639cdcef4d9f4fdb8c1b03df"}
Jul 15 21:29:51.964627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:51.964434    2580 scope.go:117] "RemoveContainer" containerID="9adc8c00a58db1e4087bd685d35dabf2f68cf267eb6f8f2c3420c13cc89fa601"
Jul 15 21:29:52.971280 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:52.971178    2580 generic.go:356] "Generic (PLEG): container finished" podID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerID="8a812ad01e0b316c798aa2987252759a7233745e6364bdc1eff7aa52f38605e0" exitCode=7
Jul 15 21:29:52.971280 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:52.971263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf" event={"ID":"998cb43f-088d-4c4f-bfb8-275b643ba1f7","Type":"ContainerDied","Data":"8a812ad01e0b316c798aa2987252759a7233745e6364bdc1eff7aa52f38605e0"}
Jul 15 21:29:52.971703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:52.971349    2580 scope.go:117] "RemoveContainer" containerID="9adc8c00a58db1e4087bd685d35dabf2f68cf267eb6f8f2c3420c13cc89fa601"
Jul 15 21:29:52.971703 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:52.971450    2580 scope.go:117] "RemoveContainer" containerID="8a812ad01e0b316c798aa2987252759a7233745e6364bdc1eff7aa52f38605e0"
Jul 15 21:29:52.971703 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:29:52.971646    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf_odh-ai-gateway-infra(998cb43f-088d-4c4f-bfb8-275b643ba1f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7"
Jul 15 21:29:53.950853 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:53.950820    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:29:53.978702 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:29:53.978675    2580 scope.go:117] "RemoveContainer" containerID="8a812ad01e0b316c798aa2987252759a7233745e6364bdc1eff7aa52f38605e0"
Jul 15 21:29:53.979061 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:29:53.978876    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf_odh-ai-gateway-infra(998cb43f-088d-4c4f-bfb8-275b643ba1f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7"
Jul 15 21:30:00.173061 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.173021    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm"]
Jul 15 21:30:00.177979 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.177940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm"
Jul 15 21:30:00.178733 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.178710    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd"]
Jul 15 21:30:00.181982 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.181952    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd"
Jul 15 21:30:00.188095 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.188078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm"
Jul 15 21:30:00.195245 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.195225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd"
Jul 15 21:30:00.210401 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.210299    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm"]
Jul 15 21:30:00.213112 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.213085    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd"]
Jul 15 21:30:00.344715 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.344684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm"]
Jul 15 21:30:00.346872 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:30:00.346840    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e395eb_f560_4445_9c11_b2b562a9f8f7.slice/crio-dfbf7d103422f6639b7c7346ee026c3aae68f1a039312785841afae7c6321fe5 WatchSource:0}: Error finding container dfbf7d103422f6639b7c7346ee026c3aae68f1a039312785841afae7c6321fe5: Status 404 returned error can't find the container with id dfbf7d103422f6639b7c7346ee026c3aae68f1a039312785841afae7c6321fe5
Jul 15 21:30:00.364937 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:00.364913    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd"]
Jul 15 21:30:00.367340 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:30:00.367309    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0768672c_84ad_425b_8556_08285c43e0bf.slice/crio-a32fa77e727b2877475dce678ad724a0e40613db31fce8fa63e106eabf8c2d1a WatchSource:0}: Error finding container a32fa77e727b2877475dce678ad724a0e40613db31fce8fa63e106eabf8c2d1a: Status 404 returned error can't find the container with id a32fa77e727b2877475dce678ad724a0e40613db31fce8fa63e106eabf8c2d1a
Jul 15 21:30:01.015856 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:01.015745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd" event={"ID":"0768672c-84ad-425b-8556-08285c43e0bf","Type":"ContainerStarted","Data":"26028be8435bdff8c16215930109010ee9fe4a36d7b34c7e4960115c4a3f9e82"}
Jul 15 21:30:01.015856 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:01.015818    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd" event={"ID":"0768672c-84ad-425b-8556-08285c43e0bf","Type":"ContainerStarted","Data":"a32fa77e727b2877475dce678ad724a0e40613db31fce8fa63e106eabf8c2d1a"}
Jul 15 21:30:01.017230 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:01.017204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm" event={"ID":"74e395eb-f560-4445-9c11-b2b562a9f8f7","Type":"ContainerStarted","Data":"e7c4c233304965149d48db8b5b30da7e685e9681e6c2754d6260ac753ddc8757"}
Jul 15 21:30:01.017317 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:01.017245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm" event={"ID":"74e395eb-f560-4445-9c11-b2b562a9f8f7","Type":"ContainerStarted","Data":"dfbf7d103422f6639b7c7346ee026c3aae68f1a039312785841afae7c6321fe5"}
Jul 15 21:30:01.033264 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:01.033211    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd" podStartSLOduration=1.033193048 podStartE2EDuration="1.033193048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:01.031508372 +0000 UTC m=+1670.388426964" watchObservedRunningTime="2026-07-15 21:30:01.033193048 +0000 UTC m=+1670.390111662"
Jul 15 21:30:01.048206 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:01.048152    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm" podStartSLOduration=1.048134251 podStartE2EDuration="1.048134251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:30:01.046172997 +0000 UTC m=+1670.403091594" watchObservedRunningTime="2026-07-15 21:30:01.048134251 +0000 UTC m=+1670.405052869"
Jul 15 21:30:02.023399 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:02.023365    2580 generic.go:356] "Generic (PLEG): container finished" podID="0768672c-84ad-425b-8556-08285c43e0bf" containerID="26028be8435bdff8c16215930109010ee9fe4a36d7b34c7e4960115c4a3f9e82" exitCode=0
Jul 15 21:30:02.023817 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:02.023442    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd" event={"ID":"0768672c-84ad-425b-8556-08285c43e0bf","Type":"ContainerDied","Data":"26028be8435bdff8c16215930109010ee9fe4a36d7b34c7e4960115c4a3f9e82"}
Jul 15 21:30:02.024806 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:02.024785    2580 generic.go:356] "Generic (PLEG): container finished" podID="74e395eb-f560-4445-9c11-b2b562a9f8f7" containerID="e7c4c233304965149d48db8b5b30da7e685e9681e6c2754d6260ac753ddc8757" exitCode=0
Jul 15 21:30:02.024892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:02.024869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm" event={"ID":"74e395eb-f560-4445-9c11-b2b562a9f8f7","Type":"ContainerDied","Data":"e7c4c233304965149d48db8b5b30da7e685e9681e6c2754d6260ac753ddc8757"}
Jul 15 21:30:03.171758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:03.171731    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm"
Jul 15 21:30:03.214371 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:03.214343    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd"
Jul 15 21:30:04.039581 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:04.039553    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd"
Jul 15 21:30:04.039774 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:04.039552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd" event={"ID":"0768672c-84ad-425b-8556-08285c43e0bf","Type":"ContainerDied","Data":"a32fa77e727b2877475dce678ad724a0e40613db31fce8fa63e106eabf8c2d1a"}
Jul 15 21:30:04.039774 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:04.039670    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a32fa77e727b2877475dce678ad724a0e40613db31fce8fa63e106eabf8c2d1a"
Jul 15 21:30:04.040787 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:04.040759    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm"
Jul 15 21:30:04.040887 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:04.040804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735850-6cwtm" event={"ID":"74e395eb-f560-4445-9c11-b2b562a9f8f7","Type":"ContainerDied","Data":"dfbf7d103422f6639b7c7346ee026c3aae68f1a039312785841afae7c6321fe5"}
Jul 15 21:30:04.040887 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:04.040841    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfbf7d103422f6639b7c7346ee026c3aae68f1a039312785841afae7c6321fe5"
Jul 15 21:30:05.128790 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:05.128761    2580 scope.go:117] "RemoveContainer" containerID="8a812ad01e0b316c798aa2987252759a7233745e6364bdc1eff7aa52f38605e0"
Jul 15 21:30:06.052291 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:06.052258    2580 generic.go:356] "Generic (PLEG): container finished" podID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerID="342c001d79675c182631d06d8dfa3b757f8c4110fd32961b448ee87fc45512bf" exitCode=0
Jul 15 21:30:06.052474 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:06.052332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf" event={"ID":"998cb43f-088d-4c4f-bfb8-275b643ba1f7","Type":"ContainerDied","Data":"342c001d79675c182631d06d8dfa3b757f8c4110fd32961b448ee87fc45512bf"}
Jul 15 21:30:06.052474 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:06.052391    2580 scope.go:117] "RemoveContainer" containerID="8a812ad01e0b316c798aa2987252759a7233745e6364bdc1eff7aa52f38605e0"
Jul 15 21:30:07.098619 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:07.098572    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"]
Jul 15 21:30:07.193921 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:07.193893    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"
Jul 15 21:30:07.333348 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:07.333240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/998cb43f-088d-4c4f-bfb8-275b643ba1f7-maas-api-service-ca\") pod \"998cb43f-088d-4c4f-bfb8-275b643ba1f7\" (UID: \"998cb43f-088d-4c4f-bfb8-275b643ba1f7\") "
Jul 15 21:30:07.333680 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:07.333656    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/998cb43f-088d-4c4f-bfb8-275b643ba1f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "998cb43f-088d-4c4f-bfb8-275b643ba1f7" (UID: "998cb43f-088d-4c4f-bfb8-275b643ba1f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:30:07.437947 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:07.435117    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/998cb43f-088d-4c4f-bfb8-275b643ba1f7-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:30:08.064500 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:08.064467    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"
Jul 15 21:30:08.064671 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:08.064502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf" event={"ID":"998cb43f-088d-4c4f-bfb8-275b643ba1f7","Type":"ContainerDied","Data":"4d1201b6665cde6fe2235088126108a5cf5b5ef7639cdcef4d9f4fdb8c1b03df"}
Jul 15 21:30:08.064671 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:08.064578    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d1201b6665cde6fe2235088126108a5cf5b5ef7639cdcef4d9f4fdb8c1b03df"
Jul 15 21:30:08.147034 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:08.147004    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"]
Jul 15 21:30:08.148763 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:08.148741    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-hb9cf"]
Jul 15 21:30:09.134493 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:09.134459    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" path="/var/lib/kubelet/pods/998cb43f-088d-4c4f-bfb8-275b643ba1f7/volumes"
Jul 15 21:30:40.555597 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.555558    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"]
Jul 15 21:30:40.556094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.555953    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerName="revoke-keys"
Jul 15 21:30:40.556094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.555983    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerName="revoke-keys"
Jul 15 21:30:40.556094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.555995    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0768672c-84ad-425b-8556-08285c43e0bf" containerName="cleanup"
Jul 15 21:30:40.556094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556001    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0768672c-84ad-425b-8556-08285c43e0bf" containerName="cleanup"
Jul 15 21:30:40.556094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556011    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerName="revoke-keys"
Jul 15 21:30:40.556094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556017    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerName="revoke-keys"
Jul 15 21:30:40.556094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556032    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74e395eb-f560-4445-9c11-b2b562a9f8f7" containerName="cleanup"
Jul 15 21:30:40.556094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556038    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="74e395eb-f560-4445-9c11-b2b562a9f8f7" containerName="cleanup"
Jul 15 21:30:40.556094 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556098    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="74e395eb-f560-4445-9c11-b2b562a9f8f7" containerName="cleanup"
Jul 15 21:30:40.556363 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556106    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerName="revoke-keys"
Jul 15 21:30:40.556363 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556111    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerName="revoke-keys"
Jul 15 21:30:40.556363 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556118    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerName="revoke-keys"
Jul 15 21:30:40.556363 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.556129    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0768672c-84ad-425b-8556-08285c43e0bf" containerName="cleanup"
Jul 15 21:30:40.559823 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.559803    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"
Jul 15 21:30:40.563110 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.563089    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:30:40.567980 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.567933    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"]
Jul 15 21:30:40.629662 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.629625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade0dc96-519d-444e-bfbe-fcf196e9b568-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9\" (UID: \"ade0dc96-519d-444e-bfbe-fcf196e9b568\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"
Jul 15 21:30:40.731172 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.731134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade0dc96-519d-444e-bfbe-fcf196e9b568-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9\" (UID: \"ade0dc96-519d-444e-bfbe-fcf196e9b568\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"
Jul 15 21:30:40.731768 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.731747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade0dc96-519d-444e-bfbe-fcf196e9b568-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9\" (UID: \"ade0dc96-519d-444e-bfbe-fcf196e9b568\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"
Jul 15 21:30:40.871390 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:40.871304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"
Jul 15 21:30:41.006147 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:41.006121    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"]
Jul 15 21:30:41.008526 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:30:41.008500    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade0dc96_519d_444e_bfbe_fcf196e9b568.slice/crio-b97605017921b375d203d995980c07d86ddfe76d41c0fd9a1d0b2eee0e827401 WatchSource:0}: Error finding container b97605017921b375d203d995980c07d86ddfe76d41c0fd9a1d0b2eee0e827401: Status 404 returned error can't find the container with id b97605017921b375d203d995980c07d86ddfe76d41c0fd9a1d0b2eee0e827401
Jul 15 21:30:41.235405 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:41.235322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9" event={"ID":"ade0dc96-519d-444e-bfbe-fcf196e9b568","Type":"ContainerStarted","Data":"b97605017921b375d203d995980c07d86ddfe76d41c0fd9a1d0b2eee0e827401"}
Jul 15 21:30:42.241314 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:42.241280    2580 generic.go:356] "Generic (PLEG): container finished" podID="ade0dc96-519d-444e-bfbe-fcf196e9b568" containerID="6218c7be3f628efa4c5494ba43953894ebac8a58c57351f5ae9f26c393abf351" exitCode=0
Jul 15 21:30:42.241697 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:42.241321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9" event={"ID":"ade0dc96-519d-444e-bfbe-fcf196e9b568","Type":"ContainerDied","Data":"6218c7be3f628efa4c5494ba43953894ebac8a58c57351f5ae9f26c393abf351"}
Jul 15 21:30:43.378298 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:43.378269    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"
Jul 15 21:30:43.454257 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:43.454221    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade0dc96-519d-444e-bfbe-fcf196e9b568-maas-api-service-ca\") pod \"ade0dc96-519d-444e-bfbe-fcf196e9b568\" (UID: \"ade0dc96-519d-444e-bfbe-fcf196e9b568\") "
Jul 15 21:30:43.454587 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:43.454566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ade0dc96-519d-444e-bfbe-fcf196e9b568-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ade0dc96-519d-444e-bfbe-fcf196e9b568" (UID: "ade0dc96-519d-444e-bfbe-fcf196e9b568"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:30:43.555636 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:43.555603    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade0dc96-519d-444e-bfbe-fcf196e9b568-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:30:44.252416 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:44.252385    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9"
Jul 15 21:30:44.252589 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:44.252432    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9" event={"ID":"ade0dc96-519d-444e-bfbe-fcf196e9b568","Type":"ContainerDied","Data":"b97605017921b375d203d995980c07d86ddfe76d41c0fd9a1d0b2eee0e827401"}
Jul 15 21:30:44.252589 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:44.252478    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b97605017921b375d203d995980c07d86ddfe76d41c0fd9a1d0b2eee0e827401"
Jul 15 21:30:51.846334 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:51.846295    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"]
Jul 15 21:30:51.846835 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:51.846545    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f" podUID="c9baf188-3b96-472b-ba9e-0dc4cce534ac" containerName="maas-api" containerID="cri-o://5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448" gracePeriod=30
Jul 15 21:30:52.096522 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.096465    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:30:52.238861 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.238821    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9baf188-3b96-472b-ba9e-0dc4cce534ac-openshift-service-ca\") pod \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") "
Jul 15 21:30:52.239091 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.238909    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86zmk\" (UniqueName: \"kubernetes.io/projected/c9baf188-3b96-472b-ba9e-0dc4cce534ac-kube-api-access-86zmk\") pod \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") "
Jul 15 21:30:52.239091 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.238954    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9baf188-3b96-472b-ba9e-0dc4cce534ac-maas-api-tls\") pod \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\" (UID: \"c9baf188-3b96-472b-ba9e-0dc4cce534ac\") "
Jul 15 21:30:52.239313 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.239286    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9baf188-3b96-472b-ba9e-0dc4cce534ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9baf188-3b96-472b-ba9e-0dc4cce534ac" (UID: "c9baf188-3b96-472b-ba9e-0dc4cce534ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:30:52.241132 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.241105    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9baf188-3b96-472b-ba9e-0dc4cce534ac-kube-api-access-86zmk" (OuterVolumeSpecName: "kube-api-access-86zmk") pod "c9baf188-3b96-472b-ba9e-0dc4cce534ac" (UID: "c9baf188-3b96-472b-ba9e-0dc4cce534ac"). InnerVolumeSpecName "kube-api-access-86zmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:30:52.241226 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.241125    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9baf188-3b96-472b-ba9e-0dc4cce534ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9baf188-3b96-472b-ba9e-0dc4cce534ac" (UID: "c9baf188-3b96-472b-ba9e-0dc4cce534ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:30:52.292256 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.292223    2580 generic.go:356] "Generic (PLEG): container finished" podID="c9baf188-3b96-472b-ba9e-0dc4cce534ac" containerID="5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448" exitCode=0
Jul 15 21:30:52.292454 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.292283    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"
Jul 15 21:30:52.292454 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.292312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f" event={"ID":"c9baf188-3b96-472b-ba9e-0dc4cce534ac","Type":"ContainerDied","Data":"5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448"}
Jul 15 21:30:52.292454 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.292367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f" event={"ID":"c9baf188-3b96-472b-ba9e-0dc4cce534ac","Type":"ContainerDied","Data":"c1def50d58a4471a35f51b18129b4a556f0d1188b6eefd5964280a1232e73694"}
Jul 15 21:30:52.292454 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.292395    2580 scope.go:117] "RemoveContainer" containerID="5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448"
Jul 15 21:30:52.301220 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.301202    2580 scope.go:117] "RemoveContainer" containerID="5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448"
Jul 15 21:30:52.301490 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:30:52.301469    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448\": container with ID starting with 5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448 not found: ID does not exist" containerID="5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448"
Jul 15 21:30:52.301538 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.301498    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448"} err="failed to get container status \"5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448\": rpc error: code = NotFound desc = could not find container \"5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448\": container with ID starting with 5adc5b8f02cd752f1dbeebbb54fcc3d7d47fd37be67f2f11fd64d44e3e30d448 not found: ID does not exist"
Jul 15 21:30:52.340626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.340591    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86zmk\" (UniqueName: \"kubernetes.io/projected/c9baf188-3b96-472b-ba9e-0dc4cce534ac-kube-api-access-86zmk\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:30:52.340626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.340620    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9baf188-3b96-472b-ba9e-0dc4cce534ac-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:30:52.340626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:52.340631    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9baf188-3b96-472b-ba9e-0dc4cce534ac-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:30:56.983323 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:56.983280    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"]
Jul 15 21:30:59.144366 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:30:59.144334    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8cee6bf5-5c4fcbfb9f-xpd8f"]
Jul 15 21:31:00.525870 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:00.525836    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd"]
Jul 15 21:31:00.532520 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:00.532493    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8cee6bf5-29735850-6bxjd"]
Jul 15 21:31:01.134671 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:01.134636    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0768672c-84ad-425b-8556-08285c43e0bf" path="/var/lib/kubelet/pods/0768672c-84ad-425b-8556-08285c43e0bf/volumes"
Jul 15 21:31:01.135020 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:01.135007    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9baf188-3b96-472b-ba9e-0dc4cce534ac" path="/var/lib/kubelet/pods/c9baf188-3b96-472b-ba9e-0dc4cce534ac/volumes"
Jul 15 21:31:14.269950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.269905    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"]
Jul 15 21:31:14.270504 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.270485    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade0dc96-519d-444e-bfbe-fcf196e9b568" containerName="revoke-keys"
Jul 15 21:31:14.270560 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.270508    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade0dc96-519d-444e-bfbe-fcf196e9b568" containerName="revoke-keys"
Jul 15 21:31:14.270560 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.270542    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerName="revoke-keys"
Jul 15 21:31:14.270560 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.270551    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="998cb43f-088d-4c4f-bfb8-275b643ba1f7" containerName="revoke-keys"
Jul 15 21:31:14.270649 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.270573    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9baf188-3b96-472b-ba9e-0dc4cce534ac" containerName="maas-api"
Jul 15 21:31:14.270649 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.270581    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9baf188-3b96-472b-ba9e-0dc4cce534ac" containerName="maas-api"
Jul 15 21:31:14.270708 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.270673    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9baf188-3b96-472b-ba9e-0dc4cce534ac" containerName="maas-api"
Jul 15 21:31:14.270708 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.270689    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ade0dc96-519d-444e-bfbe-fcf196e9b568" containerName="revoke-keys"
Jul 15 21:31:14.273928 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.273905    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.276606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.276577    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-18bc3701-serving-cert\""
Jul 15 21:31:14.276787 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.276757    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hxcvg\""
Jul 15 21:31:14.289460 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.289438    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"]
Jul 15 21:31:14.320833 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.320797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-maas-api-tls\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.321056 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.320872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsdtt\" (UniqueName: \"kubernetes.io/projected/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-kube-api-access-hsdtt\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.321056 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.320910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-openshift-service-ca\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.422274 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.422231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsdtt\" (UniqueName: \"kubernetes.io/projected/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-kube-api-access-hsdtt\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.422502 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.422289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-openshift-service-ca\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.422502 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.422379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-maas-api-tls\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.422668 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:31:14.422528    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-serving-cert: secret "maas-api-e2e-migrate-18bc3701-serving-cert" not found
Jul 15 21:31:14.422668 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:31:14.422608    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-maas-api-tls podName:96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881 nodeName:}" failed. No retries permitted until 2026-07-15 21:31:14.922585755 +0000 UTC m=+1744.279504341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-maas-api-tls") pod "maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql" (UID: "96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881") : secret "maas-api-e2e-migrate-18bc3701-serving-cert" not found
Jul 15 21:31:14.423058 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.423040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-openshift-service-ca\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.436811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.436787    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsdtt\" (UniqueName: \"kubernetes.io/projected/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-kube-api-access-hsdtt\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.927837 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.927804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-maas-api-tls\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:14.930370 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:14.930345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-maas-api-tls\") pod \"maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:15.185139 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:15.185059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:15.339044 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:15.338995    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"]
Jul 15 21:31:15.341394 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:31:15.341367    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b0e3e0_3cd1_47da_b3ef_8d3ab5f0f881.slice/crio-cc52d4547e56a4062c0d29e7abf95c7a92ef8763ef56556c472370f4742ad20e WatchSource:0}: Error finding container cc52d4547e56a4062c0d29e7abf95c7a92ef8763ef56556c472370f4742ad20e: Status 404 returned error can't find the container with id cc52d4547e56a4062c0d29e7abf95c7a92ef8763ef56556c472370f4742ad20e
Jul 15 21:31:15.342805 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:15.342784    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:31:15.408010 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:15.407969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql" event={"ID":"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881","Type":"ContainerStarted","Data":"cc52d4547e56a4062c0d29e7abf95c7a92ef8763ef56556c472370f4742ad20e"}
Jul 15 21:31:16.415027 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:16.414985    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql" event={"ID":"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881","Type":"ContainerStarted","Data":"c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f"}
Jul 15 21:31:16.415423 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:16.415062    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:16.446016 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:16.445946    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql" podStartSLOduration=1.962252775 podStartE2EDuration="2.445928558s" totalImagesPullingTime="483.675783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:31:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:31:16.44535921 +0000 UTC m=+1745.802277806" watchObservedRunningTime="2026-07-15 21:31:16.445928558 +0000 UTC m=+1745.802847154"
Jul 15 21:31:18.748971 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:18.748921    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"]
Jul 15 21:31:18.752669 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:18.752647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"
Jul 15 21:31:18.757831 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:18.757809    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:31:18.776345 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:18.776314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"]
Jul 15 21:31:18.869128 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:18.869091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8944267f-e1d7-4fdc-9b64-ffd7b36cea2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls\" (UID: \"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"
Jul 15 21:31:18.970329 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:18.970284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8944267f-e1d7-4fdc-9b64-ffd7b36cea2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls\" (UID: \"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"
Jul 15 21:31:18.970929 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:18.970907    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8944267f-e1d7-4fdc-9b64-ffd7b36cea2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls\" (UID: \"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"
Jul 15 21:31:19.062779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:19.062745    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"
Jul 15 21:31:19.199444 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:19.199312    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"]
Jul 15 21:31:19.201441 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:31:19.201416    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8944267f_e1d7_4fdc_9b64_ffd7b36cea2d.slice/crio-cdeef7d3f9ba50b5d1a9e40c608e7beb3c0d29a42cb533a9d53719cdb4fb6818 WatchSource:0}: Error finding container cdeef7d3f9ba50b5d1a9e40c608e7beb3c0d29a42cb533a9d53719cdb4fb6818: Status 404 returned error can't find the container with id cdeef7d3f9ba50b5d1a9e40c608e7beb3c0d29a42cb533a9d53719cdb4fb6818
Jul 15 21:31:19.430411 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:19.430367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls" event={"ID":"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d","Type":"ContainerStarted","Data":"cdeef7d3f9ba50b5d1a9e40c608e7beb3c0d29a42cb533a9d53719cdb4fb6818"}
Jul 15 21:31:20.436660 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:20.436626    2580 generic.go:356] "Generic (PLEG): container finished" podID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerID="6bdd7f806dcdf15c7aafd180c6fd60e900fea51be7e5a4fc9b5568167a9de9d0" exitCode=7
Jul 15 21:31:20.437091 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:20.436712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls" event={"ID":"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d","Type":"ContainerDied","Data":"6bdd7f806dcdf15c7aafd180c6fd60e900fea51be7e5a4fc9b5568167a9de9d0"}
Jul 15 21:31:20.437091 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:20.436914    2580 scope.go:117] "RemoveContainer" containerID="6bdd7f806dcdf15c7aafd180c6fd60e900fea51be7e5a4fc9b5568167a9de9d0"
Jul 15 21:31:21.444045 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:21.443929    2580 generic.go:356] "Generic (PLEG): container finished" podID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerID="a14d8309a7e8859d07f073b4e33c9def003c47848f0d0f8cb485ec3400058d39" exitCode=7
Jul 15 21:31:21.444045 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:21.443987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls" event={"ID":"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d","Type":"ContainerDied","Data":"a14d8309a7e8859d07f073b4e33c9def003c47848f0d0f8cb485ec3400058d39"}
Jul 15 21:31:21.444501 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:21.444053    2580 scope.go:117] "RemoveContainer" containerID="6bdd7f806dcdf15c7aafd180c6fd60e900fea51be7e5a4fc9b5568167a9de9d0"
Jul 15 21:31:21.444501 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:21.444202    2580 scope.go:117] "RemoveContainer" containerID="a14d8309a7e8859d07f073b4e33c9def003c47848f0d0f8cb485ec3400058d39"
Jul 15 21:31:21.444501 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:31:21.444451    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls_odh-ai-gateway-infra(8944267f-e1d7-4fdc-9b64-ffd7b36cea2d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d"
Jul 15 21:31:22.424797 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:22.424770    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:31:22.450805 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:22.450778    2580 scope.go:117] "RemoveContainer" containerID="a14d8309a7e8859d07f073b4e33c9def003c47848f0d0f8cb485ec3400058d39"
Jul 15 21:31:22.451198 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:31:22.450979    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls_odh-ai-gateway-infra(8944267f-e1d7-4fdc-9b64-ffd7b36cea2d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d"
Jul 15 21:31:37.129160 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:37.129121    2580 scope.go:117] "RemoveContainer" containerID="a14d8309a7e8859d07f073b4e33c9def003c47848f0d0f8cb485ec3400058d39"
Jul 15 21:31:37.533758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:37.533722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls" event={"ID":"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d","Type":"ContainerStarted","Data":"dcfb07bb4f8ad5ce4088733db6afb71f43dbf4850144a415fbaf667348ed3c5d"}
Jul 15 21:31:37.561315 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:37.561265    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls" podStartSLOduration=19.561250793 podStartE2EDuration="19.561250793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:31:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:31:37.558707441 +0000 UTC m=+1766.915626036" watchObservedRunningTime="2026-07-15 21:31:37.561250793 +0000 UTC m=+1766.918169389"
Jul 15 21:31:38.540135 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:38.540100    2580 generic.go:356] "Generic (PLEG): container finished" podID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerID="dcfb07bb4f8ad5ce4088733db6afb71f43dbf4850144a415fbaf667348ed3c5d" exitCode=0
Jul 15 21:31:38.540536 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:38.540176    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls" event={"ID":"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d","Type":"ContainerDied","Data":"dcfb07bb4f8ad5ce4088733db6afb71f43dbf4850144a415fbaf667348ed3c5d"}
Jul 15 21:31:38.540536 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:38.540238    2580 scope.go:117] "RemoveContainer" containerID="a14d8309a7e8859d07f073b4e33c9def003c47848f0d0f8cb485ec3400058d39"
Jul 15 21:31:38.572701 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:38.572669    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"]
Jul 15 21:31:39.686887 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:39.686853    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"
Jul 15 21:31:39.769582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:39.769547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8944267f-e1d7-4fdc-9b64-ffd7b36cea2d-maas-api-service-ca\") pod \"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d\" (UID: \"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d\") "
Jul 15 21:31:39.769942 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:39.769915    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8944267f-e1d7-4fdc-9b64-ffd7b36cea2d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" (UID: "8944267f-e1d7-4fdc-9b64-ffd7b36cea2d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:31:39.870287 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:39.870202    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8944267f-e1d7-4fdc-9b64-ffd7b36cea2d-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:31:40.553194 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:40.553160    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"
Jul 15 21:31:40.553378 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:40.553158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls" event={"ID":"8944267f-e1d7-4fdc-9b64-ffd7b36cea2d","Type":"ContainerDied","Data":"cdeef7d3f9ba50b5d1a9e40c608e7beb3c0d29a42cb533a9d53719cdb4fb6818"}
Jul 15 21:31:40.553378 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:40.553292    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdeef7d3f9ba50b5d1a9e40c608e7beb3c0d29a42cb533a9d53719cdb4fb6818"
Jul 15 21:31:40.604896 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:40.604859    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"]
Jul 15 21:31:40.610385 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:40.610354    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-wkcls"]
Jul 15 21:31:41.134059 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:31:41.134024    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" path="/var/lib/kubelet/pods/8944267f-e1d7-4fdc-9b64-ffd7b36cea2d/volumes"
Jul 15 21:32:11.181326 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:11.181198    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:32:11.189159 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:11.184849    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:32:11.189273 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:11.189233    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:32:11.192916 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:11.192895    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:32:18.829606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.829570    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"]
Jul 15 21:32:18.830031 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.829942    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerName="revoke-keys"
Jul 15 21:32:18.830031 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.829968    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerName="revoke-keys"
Jul 15 21:32:18.830031 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.829979    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerName="revoke-keys"
Jul 15 21:32:18.830031 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.829985    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerName="revoke-keys"
Jul 15 21:32:18.830168 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.830049    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerName="revoke-keys"
Jul 15 21:32:18.830168 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.830058    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerName="revoke-keys"
Jul 15 21:32:18.833039 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.833022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"
Jul 15 21:32:18.835664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.835642    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:32:18.863856 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.863809    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"]
Jul 15 21:32:18.926199 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:18.926161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d4f993-2abc-432d-aaed-59e85461ea79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l\" (UID: \"27d4f993-2abc-432d-aaed-59e85461ea79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"
Jul 15 21:32:19.027012 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:19.026933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d4f993-2abc-432d-aaed-59e85461ea79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l\" (UID: \"27d4f993-2abc-432d-aaed-59e85461ea79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"
Jul 15 21:32:19.027627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:19.027608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d4f993-2abc-432d-aaed-59e85461ea79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l\" (UID: \"27d4f993-2abc-432d-aaed-59e85461ea79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"
Jul 15 21:32:19.143376 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:19.143297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"
Jul 15 21:32:19.304206 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:19.304006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"]
Jul 15 21:32:19.307175 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:32:19.307133    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d4f993_2abc_432d_aaed_59e85461ea79.slice/crio-a24742f00dcb93e7d133521f4530847479ee80effb1af25726786541a5c58a09 WatchSource:0}: Error finding container a24742f00dcb93e7d133521f4530847479ee80effb1af25726786541a5c58a09: Status 404 returned error can't find the container with id a24742f00dcb93e7d133521f4530847479ee80effb1af25726786541a5c58a09
Jul 15 21:32:19.754035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:19.753991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l" event={"ID":"27d4f993-2abc-432d-aaed-59e85461ea79","Type":"ContainerStarted","Data":"5c4048d7962cc7b54d833190c087ab84b4a021cfe60e0d76c0ccc6348bb7ec4b"}
Jul 15 21:32:19.754212 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:19.754066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l" event={"ID":"27d4f993-2abc-432d-aaed-59e85461ea79","Type":"ContainerStarted","Data":"a24742f00dcb93e7d133521f4530847479ee80effb1af25726786541a5c58a09"}
Jul 15 21:32:19.775035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:19.774980    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l" podStartSLOduration=1.7749521430000001 podStartE2EDuration="1.774952143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:32:19.771662512 +0000 UTC m=+1809.128581109" watchObservedRunningTime="2026-07-15 21:32:19.774952143 +0000 UTC m=+1809.131870739"
Jul 15 21:32:20.761667 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:20.761629    2580 generic.go:356] "Generic (PLEG): container finished" podID="27d4f993-2abc-432d-aaed-59e85461ea79" containerID="5c4048d7962cc7b54d833190c087ab84b4a021cfe60e0d76c0ccc6348bb7ec4b" exitCode=0
Jul 15 21:32:20.762190 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:20.761711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l" event={"ID":"27d4f993-2abc-432d-aaed-59e85461ea79","Type":"ContainerDied","Data":"5c4048d7962cc7b54d833190c087ab84b4a021cfe60e0d76c0ccc6348bb7ec4b"}
Jul 15 21:32:21.905723 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:21.905694    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"
Jul 15 21:32:22.057827 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:22.057794    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d4f993-2abc-432d-aaed-59e85461ea79-maas-api-service-ca\") pod \"27d4f993-2abc-432d-aaed-59e85461ea79\" (UID: \"27d4f993-2abc-432d-aaed-59e85461ea79\") "
Jul 15 21:32:22.058237 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:22.058212    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27d4f993-2abc-432d-aaed-59e85461ea79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27d4f993-2abc-432d-aaed-59e85461ea79" (UID: "27d4f993-2abc-432d-aaed-59e85461ea79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:32:22.158768 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:22.158729    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d4f993-2abc-432d-aaed-59e85461ea79-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:32:22.773845 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:22.773815    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l"
Jul 15 21:32:22.774045 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:22.773849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l" event={"ID":"27d4f993-2abc-432d-aaed-59e85461ea79","Type":"ContainerDied","Data":"a24742f00dcb93e7d133521f4530847479ee80effb1af25726786541a5c58a09"}
Jul 15 21:32:22.774045 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:22.773900    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a24742f00dcb93e7d133521f4530847479ee80effb1af25726786541a5c58a09"
Jul 15 21:32:25.071501 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.071452    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"]
Jul 15 21:32:25.072036 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.071716    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql" podUID="96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881" containerName="maas-api" containerID="cri-o://c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f" gracePeriod=30
Jul 15 21:32:25.350714 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.350639    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:32:25.492581 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.492537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsdtt\" (UniqueName: \"kubernetes.io/projected/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-kube-api-access-hsdtt\") pod \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") "
Jul 15 21:32:25.492786 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.492593    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-maas-api-tls\") pod \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") "
Jul 15 21:32:25.492786 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.492614    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-openshift-service-ca\") pod \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\" (UID: \"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881\") "
Jul 15 21:32:25.493043 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.493016    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881" (UID: "96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:32:25.494847 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.494816    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-kube-api-access-hsdtt" (OuterVolumeSpecName: "kube-api-access-hsdtt") pod "96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881" (UID: "96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881"). InnerVolumeSpecName "kube-api-access-hsdtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:32:25.494847 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.494828    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881" (UID: "96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:32:25.594060 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.594026    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsdtt\" (UniqueName: \"kubernetes.io/projected/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-kube-api-access-hsdtt\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:32:25.594060 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.594059    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:32:25.594060 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.594069    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:32:25.791417 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.791381    2580 generic.go:356] "Generic (PLEG): container finished" podID="96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881" containerID="c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f" exitCode=0
Jul 15 21:32:25.791597 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.791421    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"
Jul 15 21:32:25.791597 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.791430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql" event={"ID":"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881","Type":"ContainerDied","Data":"c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f"}
Jul 15 21:32:25.791597 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.791479    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql" event={"ID":"96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881","Type":"ContainerDied","Data":"cc52d4547e56a4062c0d29e7abf95c7a92ef8763ef56556c472370f4742ad20e"}
Jul 15 21:32:25.791597 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.791503    2580 scope.go:117] "RemoveContainer" containerID="c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f"
Jul 15 21:32:25.800740 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.800722    2580 scope.go:117] "RemoveContainer" containerID="c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f"
Jul 15 21:32:25.801030 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:32:25.801005    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f\": container with ID starting with c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f not found: ID does not exist" containerID="c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f"
Jul 15 21:32:25.801093 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.801039    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f"} err="failed to get container status \"c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f\": rpc error: code = NotFound desc = could not find container \"c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f\": container with ID starting with c4818ddb93c831e113dd86e0eec169558b5c282acd4d9ce6812eacc4efcad32f not found: ID does not exist"
Jul 15 21:32:25.819264 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.819231    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"]
Jul 15 21:32:25.830303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:25.830276    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-18bc3701-5c964bdd47-s59ql"]
Jul 15 21:32:27.134700 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:27.134669    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881" path="/var/lib/kubelet/pods/96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881/volumes"
Jul 15 21:32:33.916663 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.916613    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"]
Jul 15 21:32:33.917276 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.917250    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27d4f993-2abc-432d-aaed-59e85461ea79" containerName="revoke-keys"
Jul 15 21:32:33.917472 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.917448    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d4f993-2abc-432d-aaed-59e85461ea79" containerName="revoke-keys"
Jul 15 21:32:33.917600 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.917587    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerName="revoke-keys"
Jul 15 21:32:33.917708 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.917690    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerName="revoke-keys"
Jul 15 21:32:33.917804 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.917721    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881" containerName="maas-api"
Jul 15 21:32:33.917804 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.917730    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881" containerName="maas-api"
Jul 15 21:32:33.917882 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.917862    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8944267f-e1d7-4fdc-9b64-ffd7b36cea2d" containerName="revoke-keys"
Jul 15 21:32:33.917882 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.917873    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="27d4f993-2abc-432d-aaed-59e85461ea79" containerName="revoke-keys"
Jul 15 21:32:33.917944 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.917883    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="96b0e3e0-3cd1-47da-b3ef-8d3ab5f0f881" containerName="maas-api"
Jul 15 21:32:33.922659 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.922632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:33.925640 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.925615    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b0fcdc21-serving-cert\""
Jul 15 21:32:33.925764 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.925743    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hxcvg\""
Jul 15 21:32:33.932789 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.932759    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"]
Jul 15 21:32:33.966809 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.966777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef15a6b2-800a-488f-8364-23180aa5b7dd-maas-api-tls\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:33.966809 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.966814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5578\" (UniqueName: \"kubernetes.io/projected/ef15a6b2-800a-488f-8364-23180aa5b7dd-kube-api-access-r5578\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:33.967090 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:33.966903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef15a6b2-800a-488f-8364-23180aa5b7dd-openshift-service-ca\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:34.067609 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:34.067562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5578\" (UniqueName: \"kubernetes.io/projected/ef15a6b2-800a-488f-8364-23180aa5b7dd-kube-api-access-r5578\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:34.067820 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:34.067671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef15a6b2-800a-488f-8364-23180aa5b7dd-openshift-service-ca\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:34.067820 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:34.067734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef15a6b2-800a-488f-8364-23180aa5b7dd-maas-api-tls\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:34.067951 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:32:34.067851    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-serving-cert: secret "maas-api-e2e-ait-b0fcdc21-serving-cert" not found
Jul 15 21:32:34.067951 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:32:34.067923    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef15a6b2-800a-488f-8364-23180aa5b7dd-maas-api-tls podName:ef15a6b2-800a-488f-8364-23180aa5b7dd nodeName:}" failed. No retries permitted until 2026-07-15 21:32:34.567903342 +0000 UTC m=+1823.924821935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ef15a6b2-800a-488f-8364-23180aa5b7dd-maas-api-tls") pod "maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs" (UID: "ef15a6b2-800a-488f-8364-23180aa5b7dd") : secret "maas-api-e2e-ait-b0fcdc21-serving-cert" not found
Jul 15 21:32:34.068355 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:34.068331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef15a6b2-800a-488f-8364-23180aa5b7dd-openshift-service-ca\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:34.079164 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:34.079132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5578\" (UniqueName: \"kubernetes.io/projected/ef15a6b2-800a-488f-8364-23180aa5b7dd-kube-api-access-r5578\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:34.573122 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:34.573082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef15a6b2-800a-488f-8364-23180aa5b7dd-maas-api-tls\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:34.575532 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:34.575497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef15a6b2-800a-488f-8364-23180aa5b7dd-maas-api-tls\") pod \"maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:34.836348 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:34.836253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:35.017011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:35.016984    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"]
Jul 15 21:32:35.019653 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:32:35.019624    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef15a6b2_800a_488f_8364_23180aa5b7dd.slice/crio-dd1d43049abc1ff66a8d29b9e70e3798b2a89c785a09e6ecd60c686eacbe081c WatchSource:0}: Error finding container dd1d43049abc1ff66a8d29b9e70e3798b2a89c785a09e6ecd60c686eacbe081c: Status 404 returned error can't find the container with id dd1d43049abc1ff66a8d29b9e70e3798b2a89c785a09e6ecd60c686eacbe081c
Jul 15 21:32:35.847045 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:35.847002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs" event={"ID":"ef15a6b2-800a-488f-8364-23180aa5b7dd","Type":"ContainerStarted","Data":"dd1d43049abc1ff66a8d29b9e70e3798b2a89c785a09e6ecd60c686eacbe081c"}
Jul 15 21:32:36.853497 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:36.853434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs" event={"ID":"ef15a6b2-800a-488f-8364-23180aa5b7dd","Type":"ContainerStarted","Data":"cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea"}
Jul 15 21:32:36.853931 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:36.853522    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:36.880870 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:36.880811    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs" podStartSLOduration=2.905635627 podStartE2EDuration="3.880793538s" totalImagesPullingTime="975.157911ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:32:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:32:36.878495364 +0000 UTC m=+1826.235413961" watchObservedRunningTime="2026-07-15 21:32:36.880793538 +0000 UTC m=+1826.237712185"
Jul 15 21:32:38.782528 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:38.782485    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"]
Jul 15 21:32:38.786238 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:38.786214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"
Jul 15 21:32:38.788844 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:38.788815    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:32:38.795112 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:38.795088    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"]
Jul 15 21:32:38.813432 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:38.813388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8896a883-6ac3-4232-909c-e88339e5fe3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp\" (UID: \"8896a883-6ac3-4232-909c-e88339e5fe3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"
Jul 15 21:32:38.914941 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:38.914902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8896a883-6ac3-4232-909c-e88339e5fe3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp\" (UID: \"8896a883-6ac3-4232-909c-e88339e5fe3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"
Jul 15 21:32:38.915584 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:38.915561    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8896a883-6ac3-4232-909c-e88339e5fe3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp\" (UID: \"8896a883-6ac3-4232-909c-e88339e5fe3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"
Jul 15 21:32:39.097536 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:39.097442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"
Jul 15 21:32:39.244446 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:39.244418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"]
Jul 15 21:32:39.246764 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:32:39.246735    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8896a883_6ac3_4232_909c_e88339e5fe3e.slice/crio-b89cac1565fbf81b74895644e41efd01d102bd8c7c99b4865feb38051b901c99 WatchSource:0}: Error finding container b89cac1565fbf81b74895644e41efd01d102bd8c7c99b4865feb38051b901c99: Status 404 returned error can't find the container with id b89cac1565fbf81b74895644e41efd01d102bd8c7c99b4865feb38051b901c99
Jul 15 21:32:39.871553 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:39.871508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp" event={"ID":"8896a883-6ac3-4232-909c-e88339e5fe3e","Type":"ContainerStarted","Data":"de3d06951c8ec183506ab80ef8620038193ea877db8e77ecc81056ab65a89ec9"}
Jul 15 21:32:39.871974 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:39.871576    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp" event={"ID":"8896a883-6ac3-4232-909c-e88339e5fe3e","Type":"ContainerStarted","Data":"b89cac1565fbf81b74895644e41efd01d102bd8c7c99b4865feb38051b901c99"}
Jul 15 21:32:39.887470 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:39.887415    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp" podStartSLOduration=1.887393087 podStartE2EDuration="1.887393087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:32:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:32:39.885807336 +0000 UTC m=+1829.242725932" watchObservedRunningTime="2026-07-15 21:32:39.887393087 +0000 UTC m=+1829.244311682"
Jul 15 21:32:40.885589 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:40.885548    2580 generic.go:356] "Generic (PLEG): container finished" podID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerID="de3d06951c8ec183506ab80ef8620038193ea877db8e77ecc81056ab65a89ec9" exitCode=7
Jul 15 21:32:40.886011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:40.885635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp" event={"ID":"8896a883-6ac3-4232-909c-e88339e5fe3e","Type":"ContainerDied","Data":"de3d06951c8ec183506ab80ef8620038193ea877db8e77ecc81056ab65a89ec9"}
Jul 15 21:32:40.886011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:40.885817    2580 scope.go:117] "RemoveContainer" containerID="de3d06951c8ec183506ab80ef8620038193ea877db8e77ecc81056ab65a89ec9"
Jul 15 21:32:41.895867 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:41.895775    2580 generic.go:356] "Generic (PLEG): container finished" podID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerID="29a04209e3376ac77545f023dd94149efc61aed4034af960277bc55c8c74324d" exitCode=7
Jul 15 21:32:41.895867 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:41.895845    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp" event={"ID":"8896a883-6ac3-4232-909c-e88339e5fe3e","Type":"ContainerDied","Data":"29a04209e3376ac77545f023dd94149efc61aed4034af960277bc55c8c74324d"}
Jul 15 21:32:41.896324 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:41.895903    2580 scope.go:117] "RemoveContainer" containerID="de3d06951c8ec183506ab80ef8620038193ea877db8e77ecc81056ab65a89ec9"
Jul 15 21:32:41.896324 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:41.896057    2580 scope.go:117] "RemoveContainer" containerID="29a04209e3376ac77545f023dd94149efc61aed4034af960277bc55c8c74324d"
Jul 15 21:32:41.896324 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:32:41.896306    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp_odh-ai-gateway-infra(8896a883-6ac3-4232-909c-e88339e5fe3e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e"
Jul 15 21:32:42.864927 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:42.864897    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:32:42.902833 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:42.902807    2580 scope.go:117] "RemoveContainer" containerID="29a04209e3376ac77545f023dd94149efc61aed4034af960277bc55c8c74324d"
Jul 15 21:32:42.903249 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:32:42.903039    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp_odh-ai-gateway-infra(8896a883-6ac3-4232-909c-e88339e5fe3e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e"
Jul 15 21:32:55.128822 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:55.128784    2580 scope.go:117] "RemoveContainer" containerID="29a04209e3376ac77545f023dd94149efc61aed4034af960277bc55c8c74324d"
Jul 15 21:32:55.969385 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:55.969350    2580 generic.go:356] "Generic (PLEG): container finished" podID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerID="f321fccae60549df79f648bdb9fa08852b4a0645f548d0e078d933119cc4b591" exitCode=0
Jul 15 21:32:55.969597 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:55.969394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp" event={"ID":"8896a883-6ac3-4232-909c-e88339e5fe3e","Type":"ContainerDied","Data":"f321fccae60549df79f648bdb9fa08852b4a0645f548d0e078d933119cc4b591"}
Jul 15 21:32:55.969597 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:55.969448    2580 scope.go:117] "RemoveContainer" containerID="29a04209e3376ac77545f023dd94149efc61aed4034af960277bc55c8c74324d"
Jul 15 21:32:57.038625 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:57.038587    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"]
Jul 15 21:32:57.112124 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:57.112100    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"
Jul 15 21:32:57.187855 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:57.187821    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8896a883-6ac3-4232-909c-e88339e5fe3e-maas-api-service-ca\") pod \"8896a883-6ac3-4232-909c-e88339e5fe3e\" (UID: \"8896a883-6ac3-4232-909c-e88339e5fe3e\") "
Jul 15 21:32:57.188232 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:57.188208    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8896a883-6ac3-4232-909c-e88339e5fe3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8896a883-6ac3-4232-909c-e88339e5fe3e" (UID: "8896a883-6ac3-4232-909c-e88339e5fe3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:32:57.288818 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:57.288785    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8896a883-6ac3-4232-909c-e88339e5fe3e-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:32:57.982580 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:57.982551    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"
Jul 15 21:32:57.982760 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:57.982583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp" event={"ID":"8896a883-6ac3-4232-909c-e88339e5fe3e","Type":"ContainerDied","Data":"b89cac1565fbf81b74895644e41efd01d102bd8c7c99b4865feb38051b901c99"}
Jul 15 21:32:57.982760 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:57.982636    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b89cac1565fbf81b74895644e41efd01d102bd8c7c99b4865feb38051b901c99"
Jul 15 21:32:58.056179 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:58.056141    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"]
Jul 15 21:32:58.064871 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:58.064836    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-ttxbp"]
Jul 15 21:32:59.134953 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:32:59.134917    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" path="/var/lib/kubelet/pods/8896a883-6ac3-4232-909c-e88339e5fe3e/volumes"
Jul 15 21:33:28.850165 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.850127    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"]
Jul 15 21:33:28.850664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.850500    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerName="revoke-keys"
Jul 15 21:33:28.850664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.850524    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerName="revoke-keys"
Jul 15 21:33:28.850664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.850539    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerName="revoke-keys"
Jul 15 21:33:28.850664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.850545    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerName="revoke-keys"
Jul 15 21:33:28.850664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.850614    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerName="revoke-keys"
Jul 15 21:33:28.850664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.850622    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerName="revoke-keys"
Jul 15 21:33:28.850664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.850631    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerName="revoke-keys"
Jul 15 21:33:28.853847 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.853831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"
Jul 15 21:33:28.856358 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.856338    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:33:28.862184 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.862159    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"]
Jul 15 21:33:28.969849 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:28.969813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf94615b-3631-4f31-a464-80b7727bbb40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7\" (UID: \"bf94615b-3631-4f31-a464-80b7727bbb40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"
Jul 15 21:33:29.071133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:29.071095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf94615b-3631-4f31-a464-80b7727bbb40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7\" (UID: \"bf94615b-3631-4f31-a464-80b7727bbb40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"
Jul 15 21:33:29.071735 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:29.071713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf94615b-3631-4f31-a464-80b7727bbb40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7\" (UID: \"bf94615b-3631-4f31-a464-80b7727bbb40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"
Jul 15 21:33:29.167284 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:29.167198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"
Jul 15 21:33:29.305558 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:29.305528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"]
Jul 15 21:33:29.307706 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:33:29.307677    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf94615b_3631_4f31_a464_80b7727bbb40.slice/crio-e9a233238be8be4f289d9653557d10495206040500b59496d879d4c1e8098db7 WatchSource:0}: Error finding container e9a233238be8be4f289d9653557d10495206040500b59496d879d4c1e8098db7: Status 404 returned error can't find the container with id e9a233238be8be4f289d9653557d10495206040500b59496d879d4c1e8098db7
Jul 15 21:33:30.152578 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:30.152544    2580 generic.go:356] "Generic (PLEG): container finished" podID="bf94615b-3631-4f31-a464-80b7727bbb40" containerID="5c9bbf87cf7cf7f458636d862ac61e3aa4d55d385102faf1cd3e69476fbb9067" exitCode=0
Jul 15 21:33:30.152989 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:30.152632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7" event={"ID":"bf94615b-3631-4f31-a464-80b7727bbb40","Type":"ContainerDied","Data":"5c9bbf87cf7cf7f458636d862ac61e3aa4d55d385102faf1cd3e69476fbb9067"}
Jul 15 21:33:30.152989 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:30.152689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7" event={"ID":"bf94615b-3631-4f31-a464-80b7727bbb40","Type":"ContainerStarted","Data":"e9a233238be8be4f289d9653557d10495206040500b59496d879d4c1e8098db7"}
Jul 15 21:33:31.289750 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:31.289727    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"
Jul 15 21:33:31.392178 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:31.392146    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf94615b-3631-4f31-a464-80b7727bbb40-maas-api-service-ca\") pod \"bf94615b-3631-4f31-a464-80b7727bbb40\" (UID: \"bf94615b-3631-4f31-a464-80b7727bbb40\") "
Jul 15 21:33:31.392556 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:31.392533    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf94615b-3631-4f31-a464-80b7727bbb40-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf94615b-3631-4f31-a464-80b7727bbb40" (UID: "bf94615b-3631-4f31-a464-80b7727bbb40"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:33:31.493438 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:31.493348    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf94615b-3631-4f31-a464-80b7727bbb40-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:33:32.164587 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:32.164554    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7"
Jul 15 21:33:32.164779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:32.164602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7" event={"ID":"bf94615b-3631-4f31-a464-80b7727bbb40","Type":"ContainerDied","Data":"e9a233238be8be4f289d9653557d10495206040500b59496d879d4c1e8098db7"}
Jul 15 21:33:32.164779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:32.164657    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9a233238be8be4f289d9653557d10495206040500b59496d879d4c1e8098db7"
Jul 15 21:33:39.642220 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:39.642185    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"]
Jul 15 21:33:39.642611 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:39.642393    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs" podUID="ef15a6b2-800a-488f-8364-23180aa5b7dd" containerName="maas-api" containerID="cri-o://cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea" gracePeriod=30
Jul 15 21:33:39.898010 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:39.897928    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:33:39.977115 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:39.977069    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5578\" (UniqueName: \"kubernetes.io/projected/ef15a6b2-800a-488f-8364-23180aa5b7dd-kube-api-access-r5578\") pod \"ef15a6b2-800a-488f-8364-23180aa5b7dd\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") "
Jul 15 21:33:39.977303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:39.977169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef15a6b2-800a-488f-8364-23180aa5b7dd-maas-api-tls\") pod \"ef15a6b2-800a-488f-8364-23180aa5b7dd\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") "
Jul 15 21:33:39.977303 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:39.977268    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef15a6b2-800a-488f-8364-23180aa5b7dd-openshift-service-ca\") pod \"ef15a6b2-800a-488f-8364-23180aa5b7dd\" (UID: \"ef15a6b2-800a-488f-8364-23180aa5b7dd\") "
Jul 15 21:33:39.977630 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:39.977599    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef15a6b2-800a-488f-8364-23180aa5b7dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef15a6b2-800a-488f-8364-23180aa5b7dd" (UID: "ef15a6b2-800a-488f-8364-23180aa5b7dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:33:39.979298 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:39.979271    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef15a6b2-800a-488f-8364-23180aa5b7dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef15a6b2-800a-488f-8364-23180aa5b7dd" (UID: "ef15a6b2-800a-488f-8364-23180aa5b7dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:33:39.979419 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:39.979395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef15a6b2-800a-488f-8364-23180aa5b7dd-kube-api-access-r5578" (OuterVolumeSpecName: "kube-api-access-r5578") pod "ef15a6b2-800a-488f-8364-23180aa5b7dd" (UID: "ef15a6b2-800a-488f-8364-23180aa5b7dd"). InnerVolumeSpecName "kube-api-access-r5578". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:33:40.078457 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.078401    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef15a6b2-800a-488f-8364-23180aa5b7dd-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:33:40.078457 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.078448    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef15a6b2-800a-488f-8364-23180aa5b7dd-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:33:40.078457 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.078462    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5578\" (UniqueName: \"kubernetes.io/projected/ef15a6b2-800a-488f-8364-23180aa5b7dd-kube-api-access-r5578\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:33:40.211419 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.211327    2580 generic.go:356] "Generic (PLEG): container finished" podID="ef15a6b2-800a-488f-8364-23180aa5b7dd" containerID="cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea" exitCode=0
Jul 15 21:33:40.211419 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.211375    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"
Jul 15 21:33:40.211419 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.211399    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs" event={"ID":"ef15a6b2-800a-488f-8364-23180aa5b7dd","Type":"ContainerDied","Data":"cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea"}
Jul 15 21:33:40.211645 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.211470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs" event={"ID":"ef15a6b2-800a-488f-8364-23180aa5b7dd","Type":"ContainerDied","Data":"dd1d43049abc1ff66a8d29b9e70e3798b2a89c785a09e6ecd60c686eacbe081c"}
Jul 15 21:33:40.211645 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.211504    2580 scope.go:117] "RemoveContainer" containerID="cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea"
Jul 15 21:33:40.221989 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.221925    2580 scope.go:117] "RemoveContainer" containerID="cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea"
Jul 15 21:33:40.222272 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:33:40.222251    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea\": container with ID starting with cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea not found: ID does not exist" containerID="cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea"
Jul 15 21:33:40.222323 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.222284    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea"} err="failed to get container status \"cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea\": rpc error: code = NotFound desc = could not find container \"cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea\": container with ID starting with cd79e2364a216e10d7794e89487cb187e655bbfc1b6a5c817f7d09f68627faea not found: ID does not exist"
Jul 15 21:33:40.236591 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.236566    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"]
Jul 15 21:33:40.242346 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:40.242322    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b0fcdc21-7b77cbb58d-7kfgs"]
Jul 15 21:33:41.133833 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:41.133800    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef15a6b2-800a-488f-8364-23180aa5b7dd" path="/var/lib/kubelet/pods/ef15a6b2-800a-488f-8364-23180aa5b7dd/volumes"
Jul 15 21:33:56.635770 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.635736    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"]
Jul 15 21:33:56.636327 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.636146    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerName="revoke-keys"
Jul 15 21:33:56.636327 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.636157    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8896a883-6ac3-4232-909c-e88339e5fe3e" containerName="revoke-keys"
Jul 15 21:33:56.636327 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.636171    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf94615b-3631-4f31-a464-80b7727bbb40" containerName="revoke-keys"
Jul 15 21:33:56.636327 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.636176    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf94615b-3631-4f31-a464-80b7727bbb40" containerName="revoke-keys"
Jul 15 21:33:56.636327 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.636189    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef15a6b2-800a-488f-8364-23180aa5b7dd" containerName="maas-api"
Jul 15 21:33:56.636327 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.636195    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef15a6b2-800a-488f-8364-23180aa5b7dd" containerName="maas-api"
Jul 15 21:33:56.636327 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.636250    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf94615b-3631-4f31-a464-80b7727bbb40" containerName="revoke-keys"
Jul 15 21:33:56.636327 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.636256    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef15a6b2-800a-488f-8364-23180aa5b7dd" containerName="maas-api"
Jul 15 21:33:56.639339 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.639315    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.642014 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.641995    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-994ab52c-serving-cert\""
Jul 15 21:33:56.642332 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.642315    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hxcvg\""
Jul 15 21:33:56.655445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.655420    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"]
Jul 15 21:33:56.735151 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.735115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-maas-api-tls\") pod \"maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.735316 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.735158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-openshift-service-ca\") pod \"maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.735316 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.735209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s64ls\" (UniqueName: \"kubernetes.io/projected/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-kube-api-access-s64ls\") pod \"maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.835720 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.835678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s64ls\" (UniqueName: \"kubernetes.io/projected/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-kube-api-access-s64ls\") pod \"maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.835921 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.835798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-maas-api-tls\") pod \"maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.835921 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.835826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-openshift-service-ca\") pod \"maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.836439 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.836419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-openshift-service-ca\") pod \"maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.838447 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.838415    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-maas-api-tls\") pod \"maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.857215 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.857182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s64ls\" (UniqueName: \"kubernetes.io/projected/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-kube-api-access-s64ls\") pod \"maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:56.949953 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:56.949870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:57.326343 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:57.326306    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"]
Jul 15 21:33:57.331670 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:33:57.331644    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a9a2e5_5917_4716_8fbd_4a898c4e923a.slice/crio-282a55a5704bb04413777c1f97da66f16e370880b794e26f8ec54b0a497ba343 WatchSource:0}: Error finding container 282a55a5704bb04413777c1f97da66f16e370880b794e26f8ec54b0a497ba343: Status 404 returned error can't find the container with id 282a55a5704bb04413777c1f97da66f16e370880b794e26f8ec54b0a497ba343
Jul 15 21:33:58.305743 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:58.305709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t" event={"ID":"f0a9a2e5-5917-4716-8fbd-4a898c4e923a","Type":"ContainerStarted","Data":"8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85"}
Jul 15 21:33:58.306165 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:58.305762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t" event={"ID":"f0a9a2e5-5917-4716-8fbd-4a898c4e923a","Type":"ContainerStarted","Data":"282a55a5704bb04413777c1f97da66f16e370880b794e26f8ec54b0a497ba343"}
Jul 15 21:33:58.306165 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:58.305802    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:33:58.357579 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:33:58.357520    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t" podStartSLOduration=1.572671887 podStartE2EDuration="2.357501129s" totalImagesPullingTime="784.829242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:33:58.354841898 +0000 UTC m=+1907.711760495" watchObservedRunningTime="2026-07-15 21:33:58.357501129 +0000 UTC m=+1907.714419727"
Jul 15 21:34:00.846156 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:00.846115    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"]
Jul 15 21:34:00.849819 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:00.849793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"
Jul 15 21:34:00.853010 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:00.852987    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:34:00.858171 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:00.858148    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"]
Jul 15 21:34:00.871310 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:00.871273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded1e71b-41ca-4e38-8b36-a4676c255b7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp\" (UID: \"ded1e71b-41ca-4e38-8b36-a4676c255b7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"
Jul 15 21:34:00.972445 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:00.972411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded1e71b-41ca-4e38-8b36-a4676c255b7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp\" (UID: \"ded1e71b-41ca-4e38-8b36-a4676c255b7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"
Jul 15 21:34:00.973139 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:00.973119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded1e71b-41ca-4e38-8b36-a4676c255b7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp\" (UID: \"ded1e71b-41ca-4e38-8b36-a4676c255b7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"
Jul 15 21:34:01.162157 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:01.162068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"
Jul 15 21:34:01.298206 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:01.298171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"]
Jul 15 21:34:01.304893 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:34:01.304852    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded1e71b_41ca_4e38_8b36_a4676c255b7a.slice/crio-4ac93529cbf19d3ecd183b3a4a327db4bfd988d97a68cc6ff5ebd40bf7a7abbe WatchSource:0}: Error finding container 4ac93529cbf19d3ecd183b3a4a327db4bfd988d97a68cc6ff5ebd40bf7a7abbe: Status 404 returned error can't find the container with id 4ac93529cbf19d3ecd183b3a4a327db4bfd988d97a68cc6ff5ebd40bf7a7abbe
Jul 15 21:34:01.325297 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:01.325257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp" event={"ID":"ded1e71b-41ca-4e38-8b36-a4676c255b7a","Type":"ContainerStarted","Data":"4ac93529cbf19d3ecd183b3a4a327db4bfd988d97a68cc6ff5ebd40bf7a7abbe"}
Jul 15 21:34:02.331665 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:02.331566    2580 generic.go:356] "Generic (PLEG): container finished" podID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerID="1d29000f8941823eb7d9fc76d9b177df91cedea8ca5163f40cb694b5c2160aa9" exitCode=7
Jul 15 21:34:02.332139 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:02.331656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp" event={"ID":"ded1e71b-41ca-4e38-8b36-a4676c255b7a","Type":"ContainerDied","Data":"1d29000f8941823eb7d9fc76d9b177df91cedea8ca5163f40cb694b5c2160aa9"}
Jul 15 21:34:02.332139 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:02.331840    2580 scope.go:117] "RemoveContainer" containerID="1d29000f8941823eb7d9fc76d9b177df91cedea8ca5163f40cb694b5c2160aa9"
Jul 15 21:34:03.338727 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:03.338691    2580 generic.go:356] "Generic (PLEG): container finished" podID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerID="84215de32006a7e94d62604b3ffde8d2a55b5a886c1502404194fee9197413a0" exitCode=7
Jul 15 21:34:03.339208 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:03.338786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp" event={"ID":"ded1e71b-41ca-4e38-8b36-a4676c255b7a","Type":"ContainerDied","Data":"84215de32006a7e94d62604b3ffde8d2a55b5a886c1502404194fee9197413a0"}
Jul 15 21:34:03.339208 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:03.338850    2580 scope.go:117] "RemoveContainer" containerID="1d29000f8941823eb7d9fc76d9b177df91cedea8ca5163f40cb694b5c2160aa9"
Jul 15 21:34:03.339208 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:03.338949    2580 scope.go:117] "RemoveContainer" containerID="84215de32006a7e94d62604b3ffde8d2a55b5a886c1502404194fee9197413a0"
Jul 15 21:34:03.339348 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:34:03.339251    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp_odh-ai-gateway-infra(ded1e71b-41ca-4e38-8b36-a4676c255b7a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a"
Jul 15 21:34:04.316789 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:04.316756    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:34:04.346024 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:04.345996    2580 scope.go:117] "RemoveContainer" containerID="84215de32006a7e94d62604b3ffde8d2a55b5a886c1502404194fee9197413a0"
Jul 15 21:34:04.346473 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:34:04.346231    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp_odh-ai-gateway-infra(ded1e71b-41ca-4e38-8b36-a4676c255b7a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a"
Jul 15 21:34:17.129184 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:17.129144    2580 scope.go:117] "RemoveContainer" containerID="84215de32006a7e94d62604b3ffde8d2a55b5a886c1502404194fee9197413a0"
Jul 15 21:34:17.416337 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:17.416257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp" event={"ID":"ded1e71b-41ca-4e38-8b36-a4676c255b7a","Type":"ContainerStarted","Data":"7b812af22d8e3ae4cd6057dce3f7fc288f4efc9008cdc1951db888e3a252cf06"}
Jul 15 21:34:17.435684 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:17.435626    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp" podStartSLOduration=17.435611942 podStartE2EDuration="17.435611942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:34:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:34:17.433471008 +0000 UTC m=+1926.790389606" watchObservedRunningTime="2026-07-15 21:34:17.435611942 +0000 UTC m=+1926.792530541"
Jul 15 21:34:18.423074 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:18.423039    2580 generic.go:356] "Generic (PLEG): container finished" podID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerID="7b812af22d8e3ae4cd6057dce3f7fc288f4efc9008cdc1951db888e3a252cf06" exitCode=0
Jul 15 21:34:18.423526 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:18.423080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp" event={"ID":"ded1e71b-41ca-4e38-8b36-a4676c255b7a","Type":"ContainerDied","Data":"7b812af22d8e3ae4cd6057dce3f7fc288f4efc9008cdc1951db888e3a252cf06"}
Jul 15 21:34:18.423526 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:18.423157    2580 scope.go:117] "RemoveContainer" containerID="84215de32006a7e94d62604b3ffde8d2a55b5a886c1502404194fee9197413a0"
Jul 15 21:34:18.445219 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:18.445192    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"]
Jul 15 21:34:19.561172 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:19.561148    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"
Jul 15 21:34:19.651859 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:19.651830    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded1e71b-41ca-4e38-8b36-a4676c255b7a-maas-api-service-ca\") pod \"ded1e71b-41ca-4e38-8b36-a4676c255b7a\" (UID: \"ded1e71b-41ca-4e38-8b36-a4676c255b7a\") "
Jul 15 21:34:19.652191 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:19.652170    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ded1e71b-41ca-4e38-8b36-a4676c255b7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ded1e71b-41ca-4e38-8b36-a4676c255b7a" (UID: "ded1e71b-41ca-4e38-8b36-a4676c255b7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:34:19.753193 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:19.753103    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ded1e71b-41ca-4e38-8b36-a4676c255b7a-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:34:20.436892 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:20.436866    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"
Jul 15 21:34:20.437115 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:20.436889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp" event={"ID":"ded1e71b-41ca-4e38-8b36-a4676c255b7a","Type":"ContainerDied","Data":"4ac93529cbf19d3ecd183b3a4a327db4bfd988d97a68cc6ff5ebd40bf7a7abbe"}
Jul 15 21:34:20.437115 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:20.436938    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ac93529cbf19d3ecd183b3a4a327db4bfd988d97a68cc6ff5ebd40bf7a7abbe"
Jul 15 21:34:20.468904 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:20.468873    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"]
Jul 15 21:34:20.474568 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:20.474527    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-4rgcp"]
Jul 15 21:34:21.134149 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:21.134113    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" path="/var/lib/kubelet/pods/ded1e71b-41ca-4e38-8b36-a4676c255b7a/volumes"
Jul 15 21:34:50.894681 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.894635    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"]
Jul 15 21:34:50.895308 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.895049    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerName="revoke-keys"
Jul 15 21:34:50.895308 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.895061    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerName="revoke-keys"
Jul 15 21:34:50.895308 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.895087    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerName="revoke-keys"
Jul 15 21:34:50.895308 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.895092    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerName="revoke-keys"
Jul 15 21:34:50.895308 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.895147    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerName="revoke-keys"
Jul 15 21:34:50.895308 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.895158    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerName="revoke-keys"
Jul 15 21:34:50.898330 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.898309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"
Jul 15 21:34:50.900988 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.900946    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:34:50.908806 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.908779    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"]
Jul 15 21:34:50.940261 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:50.940222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4675bd5-e82f-4443-8f45-3aa2b6e78159-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn\" (UID: \"c4675bd5-e82f-4443-8f45-3aa2b6e78159\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"
Jul 15 21:34:51.041013 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:51.040949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4675bd5-e82f-4443-8f45-3aa2b6e78159-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn\" (UID: \"c4675bd5-e82f-4443-8f45-3aa2b6e78159\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"
Jul 15 21:34:51.041615 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:51.041590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4675bd5-e82f-4443-8f45-3aa2b6e78159-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn\" (UID: \"c4675bd5-e82f-4443-8f45-3aa2b6e78159\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"
Jul 15 21:34:51.210332 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:51.210234    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"
Jul 15 21:34:51.346689 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:51.346617    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"]
Jul 15 21:34:51.349450 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:34:51.349423    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4675bd5_e82f_4443_8f45_3aa2b6e78159.slice/crio-c94b8ebbab0df3085ce09745d30d252c19473b01ac06cc9b2c4117faa8f47416 WatchSource:0}: Error finding container c94b8ebbab0df3085ce09745d30d252c19473b01ac06cc9b2c4117faa8f47416: Status 404 returned error can't find the container with id c94b8ebbab0df3085ce09745d30d252c19473b01ac06cc9b2c4117faa8f47416
Jul 15 21:34:51.601117 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:51.601080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn" event={"ID":"c4675bd5-e82f-4443-8f45-3aa2b6e78159","Type":"ContainerStarted","Data":"8a79afc4b7264b1209823acfdff5e7eefb6504298ce0bea4cf13869594ee2856"}
Jul 15 21:34:51.601294 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:51.601145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn" event={"ID":"c4675bd5-e82f-4443-8f45-3aa2b6e78159","Type":"ContainerStarted","Data":"c94b8ebbab0df3085ce09745d30d252c19473b01ac06cc9b2c4117faa8f47416"}
Jul 15 21:34:51.617162 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:51.617115    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn" podStartSLOduration=1.6170998060000001 podStartE2EDuration="1.617099806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:34:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:34:51.615724414 +0000 UTC m=+1960.972643027" watchObservedRunningTime="2026-07-15 21:34:51.617099806 +0000 UTC m=+1960.974018400"
Jul 15 21:34:52.607612 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:52.607580    2580 generic.go:356] "Generic (PLEG): container finished" podID="c4675bd5-e82f-4443-8f45-3aa2b6e78159" containerID="8a79afc4b7264b1209823acfdff5e7eefb6504298ce0bea4cf13869594ee2856" exitCode=0
Jul 15 21:34:52.608012 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:52.607665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn" event={"ID":"c4675bd5-e82f-4443-8f45-3aa2b6e78159","Type":"ContainerDied","Data":"8a79afc4b7264b1209823acfdff5e7eefb6504298ce0bea4cf13869594ee2856"}
Jul 15 21:34:53.743909 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:53.743880    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"
Jul 15 21:34:53.868281 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:53.868185    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4675bd5-e82f-4443-8f45-3aa2b6e78159-maas-api-service-ca\") pod \"c4675bd5-e82f-4443-8f45-3aa2b6e78159\" (UID: \"c4675bd5-e82f-4443-8f45-3aa2b6e78159\") "
Jul 15 21:34:53.868561 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:53.868535    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4675bd5-e82f-4443-8f45-3aa2b6e78159-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c4675bd5-e82f-4443-8f45-3aa2b6e78159" (UID: "c4675bd5-e82f-4443-8f45-3aa2b6e78159"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:34:53.969241 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:53.969205    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4675bd5-e82f-4443-8f45-3aa2b6e78159-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:34:54.619342 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:54.619262    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn"
Jul 15 21:34:54.619342 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:54.619296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn" event={"ID":"c4675bd5-e82f-4443-8f45-3aa2b6e78159","Type":"ContainerDied","Data":"c94b8ebbab0df3085ce09745d30d252c19473b01ac06cc9b2c4117faa8f47416"}
Jul 15 21:34:54.619342 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:34:54.619343    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c94b8ebbab0df3085ce09745d30d252c19473b01ac06cc9b2c4117faa8f47416"
Jul 15 21:35:01.388402 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.388365    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"]
Jul 15 21:35:01.389039 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.388610    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t" podUID="f0a9a2e5-5917-4716-8fbd-4a898c4e923a" containerName="maas-api" containerID="cri-o://8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85" gracePeriod=30
Jul 15 21:35:01.642371 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.642307    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:35:01.659275 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.658912    2580 generic.go:356] "Generic (PLEG): container finished" podID="f0a9a2e5-5917-4716-8fbd-4a898c4e923a" containerID="8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85" exitCode=0
Jul 15 21:35:01.659275 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.659046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t" event={"ID":"f0a9a2e5-5917-4716-8fbd-4a898c4e923a","Type":"ContainerDied","Data":"8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85"}
Jul 15 21:35:01.659275 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.659105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t" event={"ID":"f0a9a2e5-5917-4716-8fbd-4a898c4e923a","Type":"ContainerDied","Data":"282a55a5704bb04413777c1f97da66f16e370880b794e26f8ec54b0a497ba343"}
Jul 15 21:35:01.659275 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.659133    2580 scope.go:117] "RemoveContainer" containerID="8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85"
Jul 15 21:35:01.659275 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.659169    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"
Jul 15 21:35:01.669017 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.668993    2580 scope.go:117] "RemoveContainer" containerID="8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85"
Jul 15 21:35:01.669308 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:35:01.669284    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85\": container with ID starting with 8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85 not found: ID does not exist" containerID="8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85"
Jul 15 21:35:01.669365 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.669324    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85"} err="failed to get container status \"8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85\": rpc error: code = NotFound desc = could not find container \"8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85\": container with ID starting with 8987eca73c5c47262ae35ca7692180f941ef2663c7ac1c3b54e98b22bdb1fd85 not found: ID does not exist"
Jul 15 21:35:01.742155 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.742081    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s64ls\" (UniqueName: \"kubernetes.io/projected/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-kube-api-access-s64ls\") pod \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") "
Jul 15 21:35:01.742155 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.742146    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-maas-api-tls\") pod \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") "
Jul 15 21:35:01.742353 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.742211    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-openshift-service-ca\") pod \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\" (UID: \"f0a9a2e5-5917-4716-8fbd-4a898c4e923a\") "
Jul 15 21:35:01.742563 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.742530    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f0a9a2e5-5917-4716-8fbd-4a898c4e923a" (UID: "f0a9a2e5-5917-4716-8fbd-4a898c4e923a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:35:01.744261 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.744234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f0a9a2e5-5917-4716-8fbd-4a898c4e923a" (UID: "f0a9a2e5-5917-4716-8fbd-4a898c4e923a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:35:01.744415 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.744393    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-kube-api-access-s64ls" (OuterVolumeSpecName: "kube-api-access-s64ls") pod "f0a9a2e5-5917-4716-8fbd-4a898c4e923a" (UID: "f0a9a2e5-5917-4716-8fbd-4a898c4e923a"). InnerVolumeSpecName "kube-api-access-s64ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:35:01.843036 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.842997    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:35:01.843036 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.843027    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:35:01.843036 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.843044    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s64ls\" (UniqueName: \"kubernetes.io/projected/f0a9a2e5-5917-4716-8fbd-4a898c4e923a-kube-api-access-s64ls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:35:01.982409 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.982381    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"]
Jul 15 21:35:01.985878 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:01.985848    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-994ab52c-6fdddfdd5c-cjp6t"]
Jul 15 21:35:03.134415 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:35:03.134379    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0a9a2e5-5917-4716-8fbd-4a898c4e923a" path="/var/lib/kubelet/pods/f0a9a2e5-5917-4716-8fbd-4a898c4e923a/volumes"
Jul 15 21:36:11.276664 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:36:11.276622    2580 scope.go:117] "RemoveContainer" containerID="342c001d79675c182631d06d8dfa3b757f8c4110fd32961b448ee87fc45512bf"
Jul 15 21:36:11.285007 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:36:11.284987    2580 scope.go:117] "RemoveContainer" containerID="26028be8435bdff8c16215930109010ee9fe4a36d7b34c7e4960115c4a3f9e82"
Jul 15 21:37:01.153761 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.153728    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"]
Jul 15 21:37:01.154265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.154110    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerName="revoke-keys"
Jul 15 21:37:01.154265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.154121    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerName="revoke-keys"
Jul 15 21:37:01.154265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.154142    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0a9a2e5-5917-4716-8fbd-4a898c4e923a" containerName="maas-api"
Jul 15 21:37:01.154265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.154147    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0a9a2e5-5917-4716-8fbd-4a898c4e923a" containerName="maas-api"
Jul 15 21:37:01.154265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.154155    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4675bd5-e82f-4443-8f45-3aa2b6e78159" containerName="revoke-keys"
Jul 15 21:37:01.154265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.154161    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4675bd5-e82f-4443-8f45-3aa2b6e78159" containerName="revoke-keys"
Jul 15 21:37:01.154265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.154216    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ded1e71b-41ca-4e38-8b36-a4676c255b7a" containerName="revoke-keys"
Jul 15 21:37:01.154265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.154225    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0a9a2e5-5917-4716-8fbd-4a898c4e923a" containerName="maas-api"
Jul 15 21:37:01.154265 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.154232    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4675bd5-e82f-4443-8f45-3aa2b6e78159" containerName="revoke-keys"
Jul 15 21:37:01.157187 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.157170    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.160635 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.160617    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hxcvg\""
Jul 15 21:37:01.160738 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.160657    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0af6f2e9-serving-cert\""
Jul 15 21:37:01.185028 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.184998    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"]
Jul 15 21:37:01.237880 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.237846    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fe7171-1cba-43de-9e60-9f3f98e5c621-maas-api-tls\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.237880 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.237884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqnbp\" (UniqueName: \"kubernetes.io/projected/92fe7171-1cba-43de-9e60-9f3f98e5c621-kube-api-access-lqnbp\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.238111 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.238023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fe7171-1cba-43de-9e60-9f3f98e5c621-openshift-service-ca\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.338495 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.338461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fe7171-1cba-43de-9e60-9f3f98e5c621-openshift-service-ca\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.338680 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.338558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fe7171-1cba-43de-9e60-9f3f98e5c621-maas-api-tls\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.338680 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.338590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqnbp\" (UniqueName: \"kubernetes.io/projected/92fe7171-1cba-43de-9e60-9f3f98e5c621-kube-api-access-lqnbp\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.338764 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:37:01.338698    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-serving-cert: secret "maas-api-e2e-mt-0af6f2e9-serving-cert" not found
Jul 15 21:37:01.338800 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:37:01.338765    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92fe7171-1cba-43de-9e60-9f3f98e5c621-maas-api-tls podName:92fe7171-1cba-43de-9e60-9f3f98e5c621 nodeName:}" failed. No retries permitted until 2026-07-15 21:37:01.838749099 +0000 UTC m=+2091.195667674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/92fe7171-1cba-43de-9e60-9f3f98e5c621-maas-api-tls") pod "maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9" (UID: "92fe7171-1cba-43de-9e60-9f3f98e5c621") : secret "maas-api-e2e-mt-0af6f2e9-serving-cert" not found
Jul 15 21:37:01.339218 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.339201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fe7171-1cba-43de-9e60-9f3f98e5c621-openshift-service-ca\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.355476 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.355443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqnbp\" (UniqueName: \"kubernetes.io/projected/92fe7171-1cba-43de-9e60-9f3f98e5c621-kube-api-access-lqnbp\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.843489 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.843447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fe7171-1cba-43de-9e60-9f3f98e5c621-maas-api-tls\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:01.846029 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:01.845998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fe7171-1cba-43de-9e60-9f3f98e5c621-maas-api-tls\") pod \"maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:02.066482 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:02.066446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:02.229818 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:02.229791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"]
Jul 15 21:37:02.231097 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:37:02.231072    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92fe7171_1cba_43de_9e60_9f3f98e5c621.slice/crio-58348703e0cc01f214096a994cc9c4b0e78f4b6688342a55f2da8564d581c057 WatchSource:0}: Error finding container 58348703e0cc01f214096a994cc9c4b0e78f4b6688342a55f2da8564d581c057: Status 404 returned error can't find the container with id 58348703e0cc01f214096a994cc9c4b0e78f4b6688342a55f2da8564d581c057
Jul 15 21:37:02.232408 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:02.232389    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:37:02.278911 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:02.278866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9" event={"ID":"92fe7171-1cba-43de-9e60-9f3f98e5c621","Type":"ContainerStarted","Data":"58348703e0cc01f214096a994cc9c4b0e78f4b6688342a55f2da8564d581c057"}
Jul 15 21:37:03.285880 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:03.285842    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9" event={"ID":"92fe7171-1cba-43de-9e60-9f3f98e5c621","Type":"ContainerStarted","Data":"17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76"}
Jul 15 21:37:03.286275 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:03.285930    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:03.303669 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:03.303616    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9" podStartSLOduration=1.672856324 podStartE2EDuration="2.303601245s" totalImagesPullingTime="630.744921ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:37:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:37:03.301245207 +0000 UTC m=+2092.658163795" watchObservedRunningTime="2026-07-15 21:37:03.303601245 +0000 UTC m=+2092.660519907"
Jul 15 21:37:09.297799 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:09.297767    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:11.214146 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:11.214030    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:37:11.219304 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:11.217252    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:37:11.223144 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:11.223123    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:37:11.225927 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:11.225913    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:37:12.837597 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:12.837560    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"]
Jul 15 21:37:12.841337 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:12.841311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"
Jul 15 21:37:12.843684 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:12.843654    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:37:12.848448 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:12.848420    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"]
Jul 15 21:37:12.947342 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:12.947306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e80285-1cf6-4f6b-b20a-4de4063591c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4\" (UID: \"57e80285-1cf6-4f6b-b20a-4de4063591c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"
Jul 15 21:37:13.048638 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:13.048601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e80285-1cf6-4f6b-b20a-4de4063591c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4\" (UID: \"57e80285-1cf6-4f6b-b20a-4de4063591c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"
Jul 15 21:37:13.049271 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:13.049248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e80285-1cf6-4f6b-b20a-4de4063591c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4\" (UID: \"57e80285-1cf6-4f6b-b20a-4de4063591c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"
Jul 15 21:37:13.155889 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:13.155790    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"
Jul 15 21:37:13.314637 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:13.314611    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"]
Jul 15 21:37:13.317182 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:37:13.317154    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e80285_1cf6_4f6b_b20a_4de4063591c0.slice/crio-e9016bd3921a70d8081ad046a748cb4fdab7066ed95b1f6d76e40fd15105c5eb WatchSource:0}: Error finding container e9016bd3921a70d8081ad046a748cb4fdab7066ed95b1f6d76e40fd15105c5eb: Status 404 returned error can't find the container with id e9016bd3921a70d8081ad046a748cb4fdab7066ed95b1f6d76e40fd15105c5eb
Jul 15 21:37:13.339909 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:13.339876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4" event={"ID":"57e80285-1cf6-4f6b-b20a-4de4063591c0","Type":"ContainerStarted","Data":"e9016bd3921a70d8081ad046a748cb4fdab7066ed95b1f6d76e40fd15105c5eb"}
Jul 15 21:37:14.346508 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:14.346415    2580 generic.go:356] "Generic (PLEG): container finished" podID="57e80285-1cf6-4f6b-b20a-4de4063591c0" containerID="210ed9427ce6fdcd0df84e29594174f29a2d5aad19f37fb2c4d1cfd85cc7eef5" exitCode=0
Jul 15 21:37:14.346872 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:14.346499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4" event={"ID":"57e80285-1cf6-4f6b-b20a-4de4063591c0","Type":"ContainerDied","Data":"210ed9427ce6fdcd0df84e29594174f29a2d5aad19f37fb2c4d1cfd85cc7eef5"}
Jul 15 21:37:15.488612 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:15.488585    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"
Jul 15 21:37:15.576816 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:15.576777    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e80285-1cf6-4f6b-b20a-4de4063591c0-maas-api-service-ca\") pod \"57e80285-1cf6-4f6b-b20a-4de4063591c0\" (UID: \"57e80285-1cf6-4f6b-b20a-4de4063591c0\") "
Jul 15 21:37:15.577166 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:15.577145    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57e80285-1cf6-4f6b-b20a-4de4063591c0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "57e80285-1cf6-4f6b-b20a-4de4063591c0" (UID: "57e80285-1cf6-4f6b-b20a-4de4063591c0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:37:15.677529 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:15.677446    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e80285-1cf6-4f6b-b20a-4de4063591c0-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:37:16.357873 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:16.357836    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4"
Jul 15 21:37:16.357873 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:16.357863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4" event={"ID":"57e80285-1cf6-4f6b-b20a-4de4063591c0","Type":"ContainerDied","Data":"e9016bd3921a70d8081ad046a748cb4fdab7066ed95b1f6d76e40fd15105c5eb"}
Jul 15 21:37:16.358115 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:16.357913    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9016bd3921a70d8081ad046a748cb4fdab7066ed95b1f6d76e40fd15105c5eb"
Jul 15 21:37:23.623208 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:23.623166    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"]
Jul 15 21:37:23.623637 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:23.623408    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9" podUID="92fe7171-1cba-43de-9e60-9f3f98e5c621" containerName="maas-api" containerID="cri-o://17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76" gracePeriod=30
Jul 15 21:37:23.872038 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:23.872008    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:24.054109 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.054079    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fe7171-1cba-43de-9e60-9f3f98e5c621-maas-api-tls\") pod \"92fe7171-1cba-43de-9e60-9f3f98e5c621\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") "
Jul 15 21:37:24.054296 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.054167    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqnbp\" (UniqueName: \"kubernetes.io/projected/92fe7171-1cba-43de-9e60-9f3f98e5c621-kube-api-access-lqnbp\") pod \"92fe7171-1cba-43de-9e60-9f3f98e5c621\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") "
Jul 15 21:37:24.054296 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.054186    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fe7171-1cba-43de-9e60-9f3f98e5c621-openshift-service-ca\") pod \"92fe7171-1cba-43de-9e60-9f3f98e5c621\" (UID: \"92fe7171-1cba-43de-9e60-9f3f98e5c621\") "
Jul 15 21:37:24.054667 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.054633    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92fe7171-1cba-43de-9e60-9f3f98e5c621-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92fe7171-1cba-43de-9e60-9f3f98e5c621" (UID: "92fe7171-1cba-43de-9e60-9f3f98e5c621"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:37:24.056314 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.056292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92fe7171-1cba-43de-9e60-9f3f98e5c621-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92fe7171-1cba-43de-9e60-9f3f98e5c621" (UID: "92fe7171-1cba-43de-9e60-9f3f98e5c621"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:37:24.056400 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.056333    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92fe7171-1cba-43de-9e60-9f3f98e5c621-kube-api-access-lqnbp" (OuterVolumeSpecName: "kube-api-access-lqnbp") pod "92fe7171-1cba-43de-9e60-9f3f98e5c621" (UID: "92fe7171-1cba-43de-9e60-9f3f98e5c621"). InnerVolumeSpecName "kube-api-access-lqnbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:37:24.155353 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.155314    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqnbp\" (UniqueName: \"kubernetes.io/projected/92fe7171-1cba-43de-9e60-9f3f98e5c621-kube-api-access-lqnbp\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:37:24.155353 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.155346    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fe7171-1cba-43de-9e60-9f3f98e5c621-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:37:24.155353 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.155356    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fe7171-1cba-43de-9e60-9f3f98e5c621-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:37:24.400028 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.399907    2580 generic.go:356] "Generic (PLEG): container finished" podID="92fe7171-1cba-43de-9e60-9f3f98e5c621" containerID="17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76" exitCode=0
Jul 15 21:37:24.400028 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.399969    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"
Jul 15 21:37:24.400028 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.399998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9" event={"ID":"92fe7171-1cba-43de-9e60-9f3f98e5c621","Type":"ContainerDied","Data":"17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76"}
Jul 15 21:37:24.400306 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.400053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9" event={"ID":"92fe7171-1cba-43de-9e60-9f3f98e5c621","Type":"ContainerDied","Data":"58348703e0cc01f214096a994cc9c4b0e78f4b6688342a55f2da8564d581c057"}
Jul 15 21:37:24.400306 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.400078    2580 scope.go:117] "RemoveContainer" containerID="17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76"
Jul 15 21:37:24.409428 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.409411    2580 scope.go:117] "RemoveContainer" containerID="17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76"
Jul 15 21:37:24.409693 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:37:24.409675    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76\": container with ID starting with 17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76 not found: ID does not exist" containerID="17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76"
Jul 15 21:37:24.409753 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.409701    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76"} err="failed to get container status \"17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76\": rpc error: code = NotFound desc = could not find container \"17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76\": container with ID starting with 17818f4d66a1d63e3b0bd0e34f0a6f88d94f3755f781d1a073047d9539124d76 not found: ID does not exist"
Jul 15 21:37:24.428586 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.428559    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"]
Jul 15 21:37:24.430400 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:24.430379    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0af6f2e9-5c54b66c49-2zhq9"]
Jul 15 21:37:25.133980 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:37:25.133924    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92fe7171-1cba-43de-9e60-9f3f98e5c621" path="/var/lib/kubelet/pods/92fe7171-1cba-43de-9e60-9f3f98e5c621/volumes"
Jul 15 21:38:01.372482 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.372436    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"]
Jul 15 21:38:01.373178 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.373152    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57e80285-1cf6-4f6b-b20a-4de4063591c0" containerName="revoke-keys"
Jul 15 21:38:01.373359 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.373338    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57e80285-1cf6-4f6b-b20a-4de4063591c0" containerName="revoke-keys"
Jul 15 21:38:01.373457 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.373369    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92fe7171-1cba-43de-9e60-9f3f98e5c621" containerName="maas-api"
Jul 15 21:38:01.373457 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.373379    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="92fe7171-1cba-43de-9e60-9f3f98e5c621" containerName="maas-api"
Jul 15 21:38:01.373563 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.373503    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="92fe7171-1cba-43de-9e60-9f3f98e5c621" containerName="maas-api"
Jul 15 21:38:01.373563 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.373518    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57e80285-1cf6-4f6b-b20a-4de4063591c0" containerName="revoke-keys"
Jul 15 21:38:01.377231 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.377213    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.379717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.379695    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-42fd7c24-serving-cert\""
Jul 15 21:38:01.379818 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.379718    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hxcvg\""
Jul 15 21:38:01.385698 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.385523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"]
Jul 15 21:38:01.487372 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.487323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750f2463-867a-4984-862c-b9bfee6a575b-maas-api-tls\") pod \"maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.487372 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.487364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750f2463-867a-4984-862c-b9bfee6a575b-openshift-service-ca\") pod \"maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.487626 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.487467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckx6s\" (UniqueName: \"kubernetes.io/projected/750f2463-867a-4984-862c-b9bfee6a575b-kube-api-access-ckx6s\") pod \"maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.588041 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.588008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750f2463-867a-4984-862c-b9bfee6a575b-maas-api-tls\") pod \"maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.588041 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.588045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750f2463-867a-4984-862c-b9bfee6a575b-openshift-service-ca\") pod \"maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.588365 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.588078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckx6s\" (UniqueName: \"kubernetes.io/projected/750f2463-867a-4984-862c-b9bfee6a575b-kube-api-access-ckx6s\") pod \"maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.588763 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.588738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750f2463-867a-4984-862c-b9bfee6a575b-openshift-service-ca\") pod \"maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.590591 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.590565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750f2463-867a-4984-862c-b9bfee6a575b-maas-api-tls\") pod \"maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.596299 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.596274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckx6s\" (UniqueName: \"kubernetes.io/projected/750f2463-867a-4984-862c-b9bfee6a575b-kube-api-access-ckx6s\") pod \"maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.689299 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.689205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:01.825114 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:01.825088    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"]
Jul 15 21:38:01.827225 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:38:01.827199    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod750f2463_867a_4984_862c_b9bfee6a575b.slice/crio-b032863e9b5b52679427d76a174850218c1fb6a5371feaf96aa57e9534347421 WatchSource:0}: Error finding container b032863e9b5b52679427d76a174850218c1fb6a5371feaf96aa57e9534347421: Status 404 returned error can't find the container with id b032863e9b5b52679427d76a174850218c1fb6a5371feaf96aa57e9534347421
Jul 15 21:38:02.607525 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:02.607491    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v" event={"ID":"750f2463-867a-4984-862c-b9bfee6a575b","Type":"ContainerStarted","Data":"bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d"}
Jul 15 21:38:02.607917 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:02.607542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v" event={"ID":"750f2463-867a-4984-862c-b9bfee6a575b","Type":"ContainerStarted","Data":"b032863e9b5b52679427d76a174850218c1fb6a5371feaf96aa57e9534347421"}
Jul 15 21:38:02.607917 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:02.607598    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:02.630550 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:02.630489    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v" podStartSLOduration=1.255611765 podStartE2EDuration="1.63047341s" totalImagesPullingTime="374.861645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:38:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:38:02.627376889 +0000 UTC m=+2151.984295486" watchObservedRunningTime="2026-07-15 21:38:02.63047341 +0000 UTC m=+2151.987392062"
Jul 15 21:38:08.618365 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:08.618335    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:11.315932 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:11.315899    2580 scope.go:117] "RemoveContainer" containerID="dcfb07bb4f8ad5ce4088733db6afb71f43dbf4850144a415fbaf667348ed3c5d"
Jul 15 21:38:33.823535 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:33.823494    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"]
Jul 15 21:38:33.827387 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:33.827362    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"
Jul 15 21:38:33.829657 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:33.829636    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:38:33.834192 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:33.834168    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"]
Jul 15 21:38:33.886419 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:33.886376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed8bdb-b9c3-4157-a85b-4051793131e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7\" (UID: \"a4ed8bdb-b9c3-4157-a85b-4051793131e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"
Jul 15 21:38:33.987297 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:33.987267    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed8bdb-b9c3-4157-a85b-4051793131e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7\" (UID: \"a4ed8bdb-b9c3-4157-a85b-4051793131e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"
Jul 15 21:38:33.987865 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:33.987845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed8bdb-b9c3-4157-a85b-4051793131e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7\" (UID: \"a4ed8bdb-b9c3-4157-a85b-4051793131e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"
Jul 15 21:38:34.140169 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:34.140079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"
Jul 15 21:38:34.277670 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:34.277634    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"]
Jul 15 21:38:34.279823 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:38:34.279789    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ed8bdb_b9c3_4157_a85b_4051793131e7.slice/crio-d786685fcecbc11c17e16dd4b5da27556144512eae52461c467f16175c2ef3ee WatchSource:0}: Error finding container d786685fcecbc11c17e16dd4b5da27556144512eae52461c467f16175c2ef3ee: Status 404 returned error can't find the container with id d786685fcecbc11c17e16dd4b5da27556144512eae52461c467f16175c2ef3ee
Jul 15 21:38:34.777139 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:34.777033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7" event={"ID":"a4ed8bdb-b9c3-4157-a85b-4051793131e7","Type":"ContainerStarted","Data":"280bf7a464a24018ce6881c2418a2ae74da62c8375021d95e583ec181f5bf5af"}
Jul 15 21:38:34.777139 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:34.777098    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7" event={"ID":"a4ed8bdb-b9c3-4157-a85b-4051793131e7","Type":"ContainerStarted","Data":"d786685fcecbc11c17e16dd4b5da27556144512eae52461c467f16175c2ef3ee"}
Jul 15 21:38:34.796928 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:34.796881    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7" podStartSLOduration=1.796864303 podStartE2EDuration="1.796864303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:38:34.794089317 +0000 UTC m=+2184.151007913" watchObservedRunningTime="2026-07-15 21:38:34.796864303 +0000 UTC m=+2184.153782942"
Jul 15 21:38:35.784153 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:35.784122    2580 generic.go:356] "Generic (PLEG): container finished" podID="a4ed8bdb-b9c3-4157-a85b-4051793131e7" containerID="280bf7a464a24018ce6881c2418a2ae74da62c8375021d95e583ec181f5bf5af" exitCode=0
Jul 15 21:38:35.784553 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:35.784204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7" event={"ID":"a4ed8bdb-b9c3-4157-a85b-4051793131e7","Type":"ContainerDied","Data":"280bf7a464a24018ce6881c2418a2ae74da62c8375021d95e583ec181f5bf5af"}
Jul 15 21:38:36.922533 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:36.922495    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"
Jul 15 21:38:37.014774 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:37.014740    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed8bdb-b9c3-4157-a85b-4051793131e7-maas-api-service-ca\") pod \"a4ed8bdb-b9c3-4157-a85b-4051793131e7\" (UID: \"a4ed8bdb-b9c3-4157-a85b-4051793131e7\") "
Jul 15 21:38:37.015155 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:37.015133    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4ed8bdb-b9c3-4157-a85b-4051793131e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4ed8bdb-b9c3-4157-a85b-4051793131e7" (UID: "a4ed8bdb-b9c3-4157-a85b-4051793131e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:38:37.116515 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:37.116427    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed8bdb-b9c3-4157-a85b-4051793131e7-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:38:37.795682 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:37.795649    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7"
Jul 15 21:38:37.795682 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:37.795660    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7" event={"ID":"a4ed8bdb-b9c3-4157-a85b-4051793131e7","Type":"ContainerDied","Data":"d786685fcecbc11c17e16dd4b5da27556144512eae52461c467f16175c2ef3ee"}
Jul 15 21:38:37.795903 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:37.795710    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d786685fcecbc11c17e16dd4b5da27556144512eae52461c467f16175c2ef3ee"
Jul 15 21:38:44.300587 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.300549    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"]
Jul 15 21:38:44.301146 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.300807    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v" podUID="750f2463-867a-4984-862c-b9bfee6a575b" containerName="maas-api" containerID="cri-o://bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d" gracePeriod=30
Jul 15 21:38:44.566540 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.566470    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:44.682523 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.682482    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750f2463-867a-4984-862c-b9bfee6a575b-openshift-service-ca\") pod \"750f2463-867a-4984-862c-b9bfee6a575b\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") "
Jul 15 21:38:44.682713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.682569    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750f2463-867a-4984-862c-b9bfee6a575b-maas-api-tls\") pod \"750f2463-867a-4984-862c-b9bfee6a575b\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") "
Jul 15 21:38:44.682713 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.682597    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckx6s\" (UniqueName: \"kubernetes.io/projected/750f2463-867a-4984-862c-b9bfee6a575b-kube-api-access-ckx6s\") pod \"750f2463-867a-4984-862c-b9bfee6a575b\" (UID: \"750f2463-867a-4984-862c-b9bfee6a575b\") "
Jul 15 21:38:44.682906 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.682876    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/750f2463-867a-4984-862c-b9bfee6a575b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "750f2463-867a-4984-862c-b9bfee6a575b" (UID: "750f2463-867a-4984-862c-b9bfee6a575b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:38:44.684620 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.684597    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/750f2463-867a-4984-862c-b9bfee6a575b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "750f2463-867a-4984-862c-b9bfee6a575b" (UID: "750f2463-867a-4984-862c-b9bfee6a575b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:38:44.684698 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.684638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/750f2463-867a-4984-862c-b9bfee6a575b-kube-api-access-ckx6s" (OuterVolumeSpecName: "kube-api-access-ckx6s") pod "750f2463-867a-4984-862c-b9bfee6a575b" (UID: "750f2463-867a-4984-862c-b9bfee6a575b"). InnerVolumeSpecName "kube-api-access-ckx6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:38:44.783825 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.783787    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750f2463-867a-4984-862c-b9bfee6a575b-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:38:44.783825 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.783818    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750f2463-867a-4984-862c-b9bfee6a575b-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:38:44.783825 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.783827    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckx6s\" (UniqueName: \"kubernetes.io/projected/750f2463-867a-4984-862c-b9bfee6a575b-kube-api-access-ckx6s\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:38:44.833783 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.833695    2580 generic.go:356] "Generic (PLEG): container finished" podID="750f2463-867a-4984-862c-b9bfee6a575b" containerID="bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d" exitCode=0
Jul 15 21:38:44.833783 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.833741    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"
Jul 15 21:38:44.833783 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.833760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v" event={"ID":"750f2463-867a-4984-862c-b9bfee6a575b","Type":"ContainerDied","Data":"bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d"}
Jul 15 21:38:44.834068 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.833823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v" event={"ID":"750f2463-867a-4984-862c-b9bfee6a575b","Type":"ContainerDied","Data":"b032863e9b5b52679427d76a174850218c1fb6a5371feaf96aa57e9534347421"}
Jul 15 21:38:44.834068 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.833846    2580 scope.go:117] "RemoveContainer" containerID="bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d"
Jul 15 21:38:44.843271 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.843254    2580 scope.go:117] "RemoveContainer" containerID="bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d"
Jul 15 21:38:44.843522 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:38:44.843505    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d\": container with ID starting with bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d not found: ID does not exist" containerID="bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d"
Jul 15 21:38:44.843563 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.843529    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d"} err="failed to get container status \"bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d\": rpc error: code = NotFound desc = could not find container \"bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d\": container with ID starting with bc202be3872ed082f3627f060ccd4978110014fb5afc5f47cfc0764af75ecf7d not found: ID does not exist"
Jul 15 21:38:44.855268 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.855244    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"]
Jul 15 21:38:44.858573 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:44.858549    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-42fd7c24-599d9b774d-wjf9v"]
Jul 15 21:38:45.134373 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:38:45.134291    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="750f2463-867a-4984-862c-b9bfee6a575b" path="/var/lib/kubelet/pods/750f2463-867a-4984-862c-b9bfee6a575b/volumes"
Jul 15 21:39:11.338747 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:39:11.338714    2580 scope.go:117] "RemoveContainer" containerID="f321fccae60549df79f648bdb9fa08852b4a0645f548d0e078d933119cc4b591"
Jul 15 21:40:41.420203 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.420163    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"]
Jul 15 21:40:41.420687 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.420596    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="750f2463-867a-4984-862c-b9bfee6a575b" containerName="maas-api"
Jul 15 21:40:41.420687 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.420610    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="750f2463-867a-4984-862c-b9bfee6a575b" containerName="maas-api"
Jul 15 21:40:41.420687 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.420619    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4ed8bdb-b9c3-4157-a85b-4051793131e7" containerName="revoke-keys"
Jul 15 21:40:41.420687 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.420625    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ed8bdb-b9c3-4157-a85b-4051793131e7" containerName="revoke-keys"
Jul 15 21:40:41.420687 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.420684    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="750f2463-867a-4984-862c-b9bfee6a575b" containerName="maas-api"
Jul 15 21:40:41.420846 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.420692    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4ed8bdb-b9c3-4157-a85b-4051793131e7" containerName="revoke-keys"
Jul 15 21:40:41.423985 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.423941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.426971 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.426936    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7c4663-serving-cert\""
Jul 15 21:40:41.427120 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.426991    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hxcvg\""
Jul 15 21:40:41.439165 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.439129    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"]
Jul 15 21:40:41.445852 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.445823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcrx2\" (UniqueName: \"kubernetes.io/projected/286b6844-4228-4cd8-a15e-a1ec0410a50d-kube-api-access-wcrx2\") pod \"maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.446029 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.445866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/286b6844-4228-4cd8-a15e-a1ec0410a50d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.446029 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.445970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/286b6844-4228-4cd8-a15e-a1ec0410a50d-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.546443 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.546399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcrx2\" (UniqueName: \"kubernetes.io/projected/286b6844-4228-4cd8-a15e-a1ec0410a50d-kube-api-access-wcrx2\") pod \"maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.546644 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.546457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/286b6844-4228-4cd8-a15e-a1ec0410a50d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.546644 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.546498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/286b6844-4228-4cd8-a15e-a1ec0410a50d-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.547203 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.547178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/286b6844-4228-4cd8-a15e-a1ec0410a50d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.549160 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.549135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/286b6844-4228-4cd8-a15e-a1ec0410a50d-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.564099 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.564062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcrx2\" (UniqueName: \"kubernetes.io/projected/286b6844-4228-4cd8-a15e-a1ec0410a50d-kube-api-access-wcrx2\") pod \"maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.734271 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.734180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:41.880012 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:41.879980    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"]
Jul 15 21:40:41.881667 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:40:41.881629    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286b6844_4228_4cd8_a15e_a1ec0410a50d.slice/crio-d26d1958a7408afe1e93b5374e2a7193ec2a15c3bef769149a430ebb2de115f3 WatchSource:0}: Error finding container d26d1958a7408afe1e93b5374e2a7193ec2a15c3bef769149a430ebb2de115f3: Status 404 returned error can't find the container with id d26d1958a7408afe1e93b5374e2a7193ec2a15c3bef769149a430ebb2de115f3
Jul 15 21:40:42.427277 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:42.427240    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr" event={"ID":"286b6844-4228-4cd8-a15e-a1ec0410a50d","Type":"ContainerStarted","Data":"bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544"}
Jul 15 21:40:42.427718 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:42.427299    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr" event={"ID":"286b6844-4228-4cd8-a15e-a1ec0410a50d","Type":"ContainerStarted","Data":"d26d1958a7408afe1e93b5374e2a7193ec2a15c3bef769149a430ebb2de115f3"}
Jul 15 21:40:42.427718 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:42.427362    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:42.449068 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:42.449015    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr" podStartSLOduration=1.049911501 podStartE2EDuration="1.448996729s" totalImagesPullingTime="399.085228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:40:42.446818546 +0000 UTC m=+2311.803737142" watchObservedRunningTime="2026-07-15 21:40:42.448996729 +0000 UTC m=+2311.805915324"
Jul 15 21:40:48.438142 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:48.438108    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:40:49.516640 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.516596    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"]
Jul 15 21:40:49.520670 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.520647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:49.523162 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.523139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c419c9-serving-cert\""
Jul 15 21:40:49.532276 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.532250    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"]
Jul 15 21:40:49.603843 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.603805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-maas-api-tls\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:49.604056 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.603864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:49.604056 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.603895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfvwd\" (UniqueName: \"kubernetes.io/projected/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-kube-api-access-zfvwd\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:49.704435 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.704401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-maas-api-tls\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:49.704634 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.704454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:49.704634 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.704478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfvwd\" (UniqueName: \"kubernetes.io/projected/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-kube-api-access-zfvwd\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:49.704634 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:40:49.704558    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-serving-cert: secret "maas-api-e2e-inf-b-c419c9-serving-cert" not found
Jul 15 21:40:49.704634 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:40:49.704621    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-maas-api-tls podName:2d03f2c5-dfca-4b9d-ac63-29f1c8353cde nodeName:}" failed. No retries permitted until 2026-07-15 21:40:50.204604258 +0000 UTC m=+2319.561522832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-maas-api-tls") pod "maas-api-e2e-inf-b-c419c9-747f79555d-22rth" (UID: "2d03f2c5-dfca-4b9d-ac63-29f1c8353cde") : secret "maas-api-e2e-inf-b-c419c9-serving-cert" not found
Jul 15 21:40:49.705170 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.705150    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:49.716384 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:49.716355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfvwd\" (UniqueName: \"kubernetes.io/projected/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-kube-api-access-zfvwd\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:50.208683 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:50.208644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-maas-api-tls\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:50.211043 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:50.211020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-maas-api-tls\") pod \"maas-api-e2e-inf-b-c419c9-747f79555d-22rth\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:50.436319 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:50.436280    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:50.577503 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:50.577478    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"]
Jul 15 21:40:50.579691 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:40:50.579662    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d03f2c5_dfca_4b9d_ac63_29f1c8353cde.slice/crio-94ab04fb321c316c6d8f361ab2dbd7d8a2bcf2f895b558d39bd8dfbc8dfd1af9 WatchSource:0}: Error finding container 94ab04fb321c316c6d8f361ab2dbd7d8a2bcf2f895b558d39bd8dfbc8dfd1af9: Status 404 returned error can't find the container with id 94ab04fb321c316c6d8f361ab2dbd7d8a2bcf2f895b558d39bd8dfbc8dfd1af9
Jul 15 21:40:51.479285 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:51.479246    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth" event={"ID":"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde","Type":"ContainerStarted","Data":"5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7"}
Jul 15 21:40:51.479499 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:51.479304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth" event={"ID":"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde","Type":"ContainerStarted","Data":"94ab04fb321c316c6d8f361ab2dbd7d8a2bcf2f895b558d39bd8dfbc8dfd1af9"}
Jul 15 21:40:51.479499 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:51.479352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:40:51.500779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:51.500732    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth" podStartSLOduration=2.071668072 podStartE2EDuration="2.50071805s" totalImagesPullingTime="429.049978ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:40:51.498436789 +0000 UTC m=+2320.855355385" watchObservedRunningTime="2026-07-15 21:40:51.50071805 +0000 UTC m=+2320.857636860"
Jul 15 21:40:57.490207 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:40:57.490175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:41:11.370655 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:11.370615    2580 scope.go:117] "RemoveContainer" containerID="7b812af22d8e3ae4cd6057dce3f7fc288f4efc9008cdc1951db888e3a252cf06"
Jul 15 21:41:43.125473 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.125433    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"]
Jul 15 21:41:43.129334 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.129306    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"
Jul 15 21:41:43.131511 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.131484    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:41:43.136921 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.136898    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"]
Jul 15 21:41:43.252600 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.252562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7b4fd3-fe47-445c-9e14-98b4291b22d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr\" (UID: \"0e7b4fd3-fe47-445c-9e14-98b4291b22d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"
Jul 15 21:41:43.353395 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.353345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7b4fd3-fe47-445c-9e14-98b4291b22d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr\" (UID: \"0e7b4fd3-fe47-445c-9e14-98b4291b22d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"
Jul 15 21:41:43.354046 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.354024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7b4fd3-fe47-445c-9e14-98b4291b22d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr\" (UID: \"0e7b4fd3-fe47-445c-9e14-98b4291b22d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"
Jul 15 21:41:43.441709 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.441633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"
Jul 15 21:41:43.580098 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.580049    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"]
Jul 15 21:41:43.586515 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:41:43.586492    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7b4fd3_fe47_445c_9e14_98b4291b22d9.slice/crio-6f7c3064f8deebcda4b54813d0ecf6d2b6524613304d483172895d38953ca634 WatchSource:0}: Error finding container 6f7c3064f8deebcda4b54813d0ecf6d2b6524613304d483172895d38953ca634: Status 404 returned error can't find the container with id 6f7c3064f8deebcda4b54813d0ecf6d2b6524613304d483172895d38953ca634
Jul 15 21:41:43.751947 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:43.751861    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr" event={"ID":"0e7b4fd3-fe47-445c-9e14-98b4291b22d9","Type":"ContainerStarted","Data":"6f7c3064f8deebcda4b54813d0ecf6d2b6524613304d483172895d38953ca634"}
Jul 15 21:41:44.758511 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:44.758427    2580 generic.go:356] "Generic (PLEG): container finished" podID="0e7b4fd3-fe47-445c-9e14-98b4291b22d9" containerID="ff8cccad76569076d032d8b38be284018f30f90780642b988445b9efda0ae81b" exitCode=0
Jul 15 21:41:44.758869 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:44.758512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr" event={"ID":"0e7b4fd3-fe47-445c-9e14-98b4291b22d9","Type":"ContainerDied","Data":"ff8cccad76569076d032d8b38be284018f30f90780642b988445b9efda0ae81b"}
Jul 15 21:41:45.898983 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:45.898940    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"
Jul 15 21:41:45.977101 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:45.977068    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7b4fd3-fe47-445c-9e14-98b4291b22d9-maas-api-service-ca\") pod \"0e7b4fd3-fe47-445c-9e14-98b4291b22d9\" (UID: \"0e7b4fd3-fe47-445c-9e14-98b4291b22d9\") "
Jul 15 21:41:45.977490 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:45.977470    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e7b4fd3-fe47-445c-9e14-98b4291b22d9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e7b4fd3-fe47-445c-9e14-98b4291b22d9" (UID: "0e7b4fd3-fe47-445c-9e14-98b4291b22d9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:41:46.078114 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:46.078073    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7b4fd3-fe47-445c-9e14-98b4291b22d9-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:41:46.770437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:46.770407    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr"
Jul 15 21:41:46.770629 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:46.770410    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr" event={"ID":"0e7b4fd3-fe47-445c-9e14-98b4291b22d9","Type":"ContainerDied","Data":"6f7c3064f8deebcda4b54813d0ecf6d2b6524613304d483172895d38953ca634"}
Jul 15 21:41:46.770629 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:46.770532    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f7c3064f8deebcda4b54813d0ecf6d2b6524613304d483172895d38953ca634"
Jul 15 21:41:53.622017 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:53.621985    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"]
Jul 15 21:41:53.622494 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:53.622173    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr" podUID="286b6844-4228-4cd8-a15e-a1ec0410a50d" containerName="maas-api" containerID="cri-o://bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544" gracePeriod=30
Jul 15 21:41:54.065529 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.065504    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:41:54.257117 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.257011    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcrx2\" (UniqueName: \"kubernetes.io/projected/286b6844-4228-4cd8-a15e-a1ec0410a50d-kube-api-access-wcrx2\") pod \"286b6844-4228-4cd8-a15e-a1ec0410a50d\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") "
Jul 15 21:41:54.257117 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.257095    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/286b6844-4228-4cd8-a15e-a1ec0410a50d-openshift-service-ca\") pod \"286b6844-4228-4cd8-a15e-a1ec0410a50d\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") "
Jul 15 21:41:54.257360 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.257254    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/286b6844-4228-4cd8-a15e-a1ec0410a50d-maas-api-tls\") pod \"286b6844-4228-4cd8-a15e-a1ec0410a50d\" (UID: \"286b6844-4228-4cd8-a15e-a1ec0410a50d\") "
Jul 15 21:41:54.257418 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.257405    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/286b6844-4228-4cd8-a15e-a1ec0410a50d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "286b6844-4228-4cd8-a15e-a1ec0410a50d" (UID: "286b6844-4228-4cd8-a15e-a1ec0410a50d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:41:54.257555 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.257535    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/286b6844-4228-4cd8-a15e-a1ec0410a50d-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:41:54.259209 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.259186    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/286b6844-4228-4cd8-a15e-a1ec0410a50d-kube-api-access-wcrx2" (OuterVolumeSpecName: "kube-api-access-wcrx2") pod "286b6844-4228-4cd8-a15e-a1ec0410a50d" (UID: "286b6844-4228-4cd8-a15e-a1ec0410a50d"). InnerVolumeSpecName "kube-api-access-wcrx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:41:54.259290 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.259234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/286b6844-4228-4cd8-a15e-a1ec0410a50d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "286b6844-4228-4cd8-a15e-a1ec0410a50d" (UID: "286b6844-4228-4cd8-a15e-a1ec0410a50d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:41:54.358687 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.358643    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/286b6844-4228-4cd8-a15e-a1ec0410a50d-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:41:54.358687 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.358674    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcrx2\" (UniqueName: \"kubernetes.io/projected/286b6844-4228-4cd8-a15e-a1ec0410a50d-kube-api-access-wcrx2\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:41:54.811870 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.811839    2580 generic.go:356] "Generic (PLEG): container finished" podID="286b6844-4228-4cd8-a15e-a1ec0410a50d" containerID="bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544" exitCode=0
Jul 15 21:41:54.812385 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.811889    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"
Jul 15 21:41:54.812385 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.811922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr" event={"ID":"286b6844-4228-4cd8-a15e-a1ec0410a50d","Type":"ContainerDied","Data":"bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544"}
Jul 15 21:41:54.812385 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.812006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr" event={"ID":"286b6844-4228-4cd8-a15e-a1ec0410a50d","Type":"ContainerDied","Data":"d26d1958a7408afe1e93b5374e2a7193ec2a15c3bef769149a430ebb2de115f3"}
Jul 15 21:41:54.812385 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.812037    2580 scope.go:117] "RemoveContainer" containerID="bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544"
Jul 15 21:41:54.821456 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.821435    2580 scope.go:117] "RemoveContainer" containerID="bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544"
Jul 15 21:41:54.821713 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:41:54.821692    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544\": container with ID starting with bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544 not found: ID does not exist" containerID="bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544"
Jul 15 21:41:54.821764 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.821721    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544"} err="failed to get container status \"bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544\": rpc error: code = NotFound desc = could not find container \"bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544\": container with ID starting with bfd42802c3e9bc66e641a7552d11875fd7d1e095bcaec7fbcdaf5092572cc544 not found: ID does not exist"
Jul 15 21:41:54.835625 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.835600    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"]
Jul 15 21:41:54.842305 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:54.842282    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c4663-5d8995d567-bt6cr"]
Jul 15 21:41:55.135059 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:41:55.134948    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="286b6844-4228-4cd8-a15e-a1ec0410a50d" path="/var/lib/kubelet/pods/286b6844-4228-4cd8-a15e-a1ec0410a50d/volumes"
Jul 15 21:42:09.410190 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.410153    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"]
Jul 15 21:42:09.410578 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.410528    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7b4fd3-fe47-445c-9e14-98b4291b22d9" containerName="revoke-keys"
Jul 15 21:42:09.410578 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.410539    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7b4fd3-fe47-445c-9e14-98b4291b22d9" containerName="revoke-keys"
Jul 15 21:42:09.410578 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.410558    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="286b6844-4228-4cd8-a15e-a1ec0410a50d" containerName="maas-api"
Jul 15 21:42:09.410578 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.410563    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="286b6844-4228-4cd8-a15e-a1ec0410a50d" containerName="maas-api"
Jul 15 21:42:09.410702 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.410627    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="286b6844-4228-4cd8-a15e-a1ec0410a50d" containerName="maas-api"
Jul 15 21:42:09.410702 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.410636    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7b4fd3-fe47-445c-9e14-98b4291b22d9" containerName="revoke-keys"
Jul 15 21:42:09.415095 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.415075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"
Jul 15 21:42:09.422344 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.422308    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:42:09.426949 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.426922    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"]
Jul 15 21:42:09.484430 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.484376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60dbd51a-0086-4764-8781-98fcdb340e51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5\" (UID: \"60dbd51a-0086-4764-8781-98fcdb340e51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"
Jul 15 21:42:09.585119 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.585083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60dbd51a-0086-4764-8781-98fcdb340e51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5\" (UID: \"60dbd51a-0086-4764-8781-98fcdb340e51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"
Jul 15 21:42:09.585758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.585738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60dbd51a-0086-4764-8781-98fcdb340e51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5\" (UID: \"60dbd51a-0086-4764-8781-98fcdb340e51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"
Jul 15 21:42:09.734404 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.734314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"
Jul 15 21:42:09.866138 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.866111    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"]
Jul 15 21:42:09.868337 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:42:09.868304    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60dbd51a_0086_4764_8781_98fcdb340e51.slice/crio-702cfe0b52ff5d7b30e35c37b29a5f0eff451c47ddd0e2d09c6ee18fd0d40cde WatchSource:0}: Error finding container 702cfe0b52ff5d7b30e35c37b29a5f0eff451c47ddd0e2d09c6ee18fd0d40cde: Status 404 returned error can't find the container with id 702cfe0b52ff5d7b30e35c37b29a5f0eff451c47ddd0e2d09c6ee18fd0d40cde
Jul 15 21:42:09.893274 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:09.893238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5" event={"ID":"60dbd51a-0086-4764-8781-98fcdb340e51","Type":"ContainerStarted","Data":"702cfe0b52ff5d7b30e35c37b29a5f0eff451c47ddd0e2d09c6ee18fd0d40cde"}
Jul 15 21:42:10.899768 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:10.899683    2580 generic.go:356] "Generic (PLEG): container finished" podID="60dbd51a-0086-4764-8781-98fcdb340e51" containerID="171d8a6027525834b1305d9e0be0d540a534935e7d6836a46e63d72c7b9ba697" exitCode=0
Jul 15 21:42:10.900151 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:10.899759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5" event={"ID":"60dbd51a-0086-4764-8781-98fcdb340e51","Type":"ContainerDied","Data":"171d8a6027525834b1305d9e0be0d540a534935e7d6836a46e63d72c7b9ba697"}
Jul 15 21:42:11.249741 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:11.249577    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:42:11.272984 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:11.252913    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:42:11.272984 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:11.258315    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:42:11.272984 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:11.261355    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:42:12.038333 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:12.038312    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"
Jul 15 21:42:12.117902 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:12.117867    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60dbd51a-0086-4764-8781-98fcdb340e51-maas-api-service-ca\") pod \"60dbd51a-0086-4764-8781-98fcdb340e51\" (UID: \"60dbd51a-0086-4764-8781-98fcdb340e51\") "
Jul 15 21:42:12.118286 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:12.118266    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60dbd51a-0086-4764-8781-98fcdb340e51-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "60dbd51a-0086-4764-8781-98fcdb340e51" (UID: "60dbd51a-0086-4764-8781-98fcdb340e51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:42:12.219643 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:12.219547    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60dbd51a-0086-4764-8781-98fcdb340e51-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:42:12.910658 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:12.910627    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5"
Jul 15 21:42:12.910843 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:12.910673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5" event={"ID":"60dbd51a-0086-4764-8781-98fcdb340e51","Type":"ContainerDied","Data":"702cfe0b52ff5d7b30e35c37b29a5f0eff451c47ddd0e2d09c6ee18fd0d40cde"}
Jul 15 21:42:12.910843 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:12.910719    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="702cfe0b52ff5d7b30e35c37b29a5f0eff451c47ddd0e2d09c6ee18fd0d40cde"
Jul 15 21:42:19.854248 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:19.854212    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"]
Jul 15 21:42:19.854776 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:19.854440    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth" podUID="2d03f2c5-dfca-4b9d-ac63-29f1c8353cde" containerName="maas-api" containerID="cri-o://5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7" gracePeriod=30
Jul 15 21:42:20.109464 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.109404    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:42:20.192603 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.192555    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-openshift-service-ca\") pod \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") "
Jul 15 21:42:20.192779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.192673    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfvwd\" (UniqueName: \"kubernetes.io/projected/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-kube-api-access-zfvwd\") pod \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") "
Jul 15 21:42:20.192779 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.192708    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-maas-api-tls\") pod \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\" (UID: \"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde\") "
Jul 15 21:42:20.193039 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.193014    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2d03f2c5-dfca-4b9d-ac63-29f1c8353cde" (UID: "2d03f2c5-dfca-4b9d-ac63-29f1c8353cde"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:42:20.194813 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.194784    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-kube-api-access-zfvwd" (OuterVolumeSpecName: "kube-api-access-zfvwd") pod "2d03f2c5-dfca-4b9d-ac63-29f1c8353cde" (UID: "2d03f2c5-dfca-4b9d-ac63-29f1c8353cde"). InnerVolumeSpecName "kube-api-access-zfvwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:42:20.194813 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.194800    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2d03f2c5-dfca-4b9d-ac63-29f1c8353cde" (UID: "2d03f2c5-dfca-4b9d-ac63-29f1c8353cde"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:42:20.293399 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.293365    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfvwd\" (UniqueName: \"kubernetes.io/projected/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-kube-api-access-zfvwd\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:42:20.293399 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.293395    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:42:20.293399 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.293405    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:42:20.953624 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.953586    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d03f2c5-dfca-4b9d-ac63-29f1c8353cde" containerID="5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7" exitCode=0
Jul 15 21:42:20.954242 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.953640    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"
Jul 15 21:42:20.954242 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.953670    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth" event={"ID":"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde","Type":"ContainerDied","Data":"5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7"}
Jul 15 21:42:20.954242 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.953731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth" event={"ID":"2d03f2c5-dfca-4b9d-ac63-29f1c8353cde","Type":"ContainerDied","Data":"94ab04fb321c316c6d8f361ab2dbd7d8a2bcf2f895b558d39bd8dfbc8dfd1af9"}
Jul 15 21:42:20.954242 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.953756    2580 scope.go:117] "RemoveContainer" containerID="5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7"
Jul 15 21:42:20.963802 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.963785    2580 scope.go:117] "RemoveContainer" containerID="5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7"
Jul 15 21:42:20.964069 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:42:20.964051    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7\": container with ID starting with 5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7 not found: ID does not exist" containerID="5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7"
Jul 15 21:42:20.964125 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.964077    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7"} err="failed to get container status \"5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7\": rpc error: code = NotFound desc = could not find container \"5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7\": container with ID starting with 5e7388d50f317fe9fe1162febf6f49d2b880a166471d9f71c6d73e5252b838b7 not found: ID does not exist"
Jul 15 21:42:20.976676 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.976653    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"]
Jul 15 21:42:20.983403 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:20.983383    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c419c9-747f79555d-22rth"]
Jul 15 21:42:21.136295 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:21.136259    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d03f2c5-dfca-4b9d-ac63-29f1c8353cde" path="/var/lib/kubelet/pods/2d03f2c5-dfca-4b9d-ac63-29f1c8353cde/volumes"
Jul 15 21:42:39.083826 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.083723    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"]
Jul 15 21:42:39.084507 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.084467    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60dbd51a-0086-4764-8781-98fcdb340e51" containerName="revoke-keys"
Jul 15 21:42:39.084507 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.084496    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="60dbd51a-0086-4764-8781-98fcdb340e51" containerName="revoke-keys"
Jul 15 21:42:39.084643 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.084528    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d03f2c5-dfca-4b9d-ac63-29f1c8353cde" containerName="maas-api"
Jul 15 21:42:39.084643 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.084538    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d03f2c5-dfca-4b9d-ac63-29f1c8353cde" containerName="maas-api"
Jul 15 21:42:39.084643 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.084630    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d03f2c5-dfca-4b9d-ac63-29f1c8353cde" containerName="maas-api"
Jul 15 21:42:39.084802 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.084646    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="60dbd51a-0086-4764-8781-98fcdb340e51" containerName="revoke-keys"
Jul 15 21:42:39.089787 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.089766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.092272 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.092250    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-652862-serving-cert\""
Jul 15 21:42:39.092542 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.092527    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hxcvg\""
Jul 15 21:42:39.101067 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.101037    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"]
Jul 15 21:42:39.281307 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.281269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d003f9-cb9b-4e79-982e-d26dbacefd05-maas-api-tls\") pod \"maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.281307 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.281311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d003f9-cb9b-4e79-982e-d26dbacefd05-openshift-service-ca\") pod \"maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.281542 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.281332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4mk2\" (UniqueName: \"kubernetes.io/projected/59d003f9-cb9b-4e79-982e-d26dbacefd05-kube-api-access-v4mk2\") pod \"maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.382146 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.382036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d003f9-cb9b-4e79-982e-d26dbacefd05-maas-api-tls\") pod \"maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.382146 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.382081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d003f9-cb9b-4e79-982e-d26dbacefd05-openshift-service-ca\") pod \"maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.382146 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.382102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4mk2\" (UniqueName: \"kubernetes.io/projected/59d003f9-cb9b-4e79-982e-d26dbacefd05-kube-api-access-v4mk2\") pod \"maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.382825 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.382793    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d003f9-cb9b-4e79-982e-d26dbacefd05-openshift-service-ca\") pod \"maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.384549 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.384528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d003f9-cb9b-4e79-982e-d26dbacefd05-maas-api-tls\") pod \"maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.391004 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.390976    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4mk2\" (UniqueName: \"kubernetes.io/projected/59d003f9-cb9b-4e79-982e-d26dbacefd05-kube-api-access-v4mk2\") pod \"maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.402731 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.402705    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:39.560116 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.560082    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"]
Jul 15 21:42:39.562635 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:42:39.562594    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d003f9_cb9b_4e79_982e_d26dbacefd05.slice/crio-483d7986a78a0e6b10782af86ea1d24d13cebbd597ccccb301b835717fd734d0 WatchSource:0}: Error finding container 483d7986a78a0e6b10782af86ea1d24d13cebbd597ccccb301b835717fd734d0: Status 404 returned error can't find the container with id 483d7986a78a0e6b10782af86ea1d24d13cebbd597ccccb301b835717fd734d0
Jul 15 21:42:39.564237 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:39.564216    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 21:42:40.066465 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:40.066431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh" event={"ID":"59d003f9-cb9b-4e79-982e-d26dbacefd05","Type":"ContainerStarted","Data":"483d7986a78a0e6b10782af86ea1d24d13cebbd597ccccb301b835717fd734d0"}
Jul 15 21:42:41.073149 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:41.073116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh" event={"ID":"59d003f9-cb9b-4e79-982e-d26dbacefd05","Type":"ContainerStarted","Data":"a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d"}
Jul 15 21:42:41.073657 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:41.073211    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:41.100147 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:41.100094    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh" podStartSLOduration=1.624958962 podStartE2EDuration="2.100075309s" totalImagesPullingTime="475.116347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:42:41.098161023 +0000 UTC m=+2430.455079619" watchObservedRunningTime="2026-07-15 21:42:41.100075309 +0000 UTC m=+2430.456993906"
Jul 15 21:42:47.084950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.084917    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:42:47.154080 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.153857    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"]
Jul 15 21:42:47.166894 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.166863    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.170095 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.170067    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-da009d-serving-cert\""
Jul 15 21:42:47.177011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.176982    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"]
Jul 15 21:42:47.264984 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.264922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7slh\" (UniqueName: \"kubernetes.io/projected/07b6e26a-81a4-43c5-bd48-6ff3002256f7-kube-api-access-q7slh\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.265184 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.265126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b6e26a-81a4-43c5-bd48-6ff3002256f7-maas-api-tls\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.265249 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.265179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b6e26a-81a4-43c5-bd48-6ff3002256f7-openshift-service-ca\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.366037 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.365909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b6e26a-81a4-43c5-bd48-6ff3002256f7-maas-api-tls\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.366037 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.365990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b6e26a-81a4-43c5-bd48-6ff3002256f7-openshift-service-ca\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.366289 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.366045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7slh\" (UniqueName: \"kubernetes.io/projected/07b6e26a-81a4-43c5-bd48-6ff3002256f7-kube-api-access-q7slh\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.366289 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:42:47.366080    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-serving-cert: secret "maas-api-e2e-api-b-da009d-serving-cert" not found
Jul 15 21:42:47.366289 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:42:47.366155    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07b6e26a-81a4-43c5-bd48-6ff3002256f7-maas-api-tls podName:07b6e26a-81a4-43c5-bd48-6ff3002256f7 nodeName:}" failed. No retries permitted until 2026-07-15 21:42:47.866132084 +0000 UTC m=+2437.223050673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07b6e26a-81a4-43c5-bd48-6ff3002256f7-maas-api-tls") pod "maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw" (UID: "07b6e26a-81a4-43c5-bd48-6ff3002256f7") : secret "maas-api-e2e-api-b-da009d-serving-cert" not found
Jul 15 21:42:47.366629 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.366608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b6e26a-81a4-43c5-bd48-6ff3002256f7-openshift-service-ca\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.378920 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.378888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7slh\" (UniqueName: \"kubernetes.io/projected/07b6e26a-81a4-43c5-bd48-6ff3002256f7-kube-api-access-q7slh\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.871238 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.871199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b6e26a-81a4-43c5-bd48-6ff3002256f7-maas-api-tls\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:47.873646 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:47.873612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b6e26a-81a4-43c5-bd48-6ff3002256f7-maas-api-tls\") pod \"maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:48.080143 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:48.080106    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:48.228212 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:48.228187    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"]
Jul 15 21:42:48.230245 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:42:48.230220    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b6e26a_81a4_43c5_bd48_6ff3002256f7.slice/crio-dfb919205192c4370417c848c32e53bfdc464188245a6c42e960a31e43c925f2 WatchSource:0}: Error finding container dfb919205192c4370417c848c32e53bfdc464188245a6c42e960a31e43c925f2: Status 404 returned error can't find the container with id dfb919205192c4370417c848c32e53bfdc464188245a6c42e960a31e43c925f2
Jul 15 21:42:49.119825 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:49.119788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw" event={"ID":"07b6e26a-81a4-43c5-bd48-6ff3002256f7","Type":"ContainerStarted","Data":"82538ff0ac2a7e128765fedc3eaf338803c797927343d4901c98b9bf1bbda4c6"}
Jul 15 21:42:49.120039 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:49.119841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw" event={"ID":"07b6e26a-81a4-43c5-bd48-6ff3002256f7","Type":"ContainerStarted","Data":"dfb919205192c4370417c848c32e53bfdc464188245a6c42e960a31e43c925f2"}
Jul 15 21:42:49.120039 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:49.119912    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:49.165225 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:49.165176    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw" podStartSLOduration=1.785280824 podStartE2EDuration="2.165160289s" totalImagesPullingTime="379.879465ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:42:49.163897444 +0000 UTC m=+2438.520816040" watchObservedRunningTime="2026-07-15 21:42:49.165160289 +0000 UTC m=+2438.522078886"
Jul 15 21:42:55.135107 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:55.135079    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:42:59.500484 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:59.500448    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"]
Jul 15 21:42:59.504329 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:59.504307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"
Jul 15 21:42:59.509904 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:59.509882    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:42:59.526761 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:59.526725    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"]
Jul 15 21:42:59.583482 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:59.583442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ce94df4-03c6-4dad-aea6-b2462d3f4633-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-652862-qhxvz\" (UID: \"4ce94df4-03c6-4dad-aea6-b2462d3f4633\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"
Jul 15 21:42:59.684866 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:59.684825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ce94df4-03c6-4dad-aea6-b2462d3f4633-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-652862-qhxvz\" (UID: \"4ce94df4-03c6-4dad-aea6-b2462d3f4633\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"
Jul 15 21:42:59.685521 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:59.685497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ce94df4-03c6-4dad-aea6-b2462d3f4633-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-652862-qhxvz\" (UID: \"4ce94df4-03c6-4dad-aea6-b2462d3f4633\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"
Jul 15 21:42:59.815246 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:59.815207    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"
Jul 15 21:42:59.954120 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:42:59.954094    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"]
Jul 15 21:42:59.955859 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:42:59.955828    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce94df4_03c6_4dad_aea6_b2462d3f4633.slice/crio-31015ae0655b828db2935e199d138a37e0f94d50b9e0269964796399f43e95bb WatchSource:0}: Error finding container 31015ae0655b828db2935e199d138a37e0f94d50b9e0269964796399f43e95bb: Status 404 returned error can't find the container with id 31015ae0655b828db2935e199d138a37e0f94d50b9e0269964796399f43e95bb
Jul 15 21:43:00.180010 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:00.179952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz" event={"ID":"4ce94df4-03c6-4dad-aea6-b2462d3f4633","Type":"ContainerStarted","Data":"31015ae0655b828db2935e199d138a37e0f94d50b9e0269964796399f43e95bb"}
Jul 15 21:43:01.186633 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:01.186598    2580 generic.go:356] "Generic (PLEG): container finished" podID="4ce94df4-03c6-4dad-aea6-b2462d3f4633" containerID="679447ee36c06b65e127c1d3f16d14790710eae55725a67ff7c7a95e466869e6" exitCode=0
Jul 15 21:43:01.187047 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:01.186685    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz" event={"ID":"4ce94df4-03c6-4dad-aea6-b2462d3f4633","Type":"ContainerDied","Data":"679447ee36c06b65e127c1d3f16d14790710eae55725a67ff7c7a95e466869e6"}
Jul 15 21:43:02.330329 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:02.330303    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"
Jul 15 21:43:02.411104 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:02.411067    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ce94df4-03c6-4dad-aea6-b2462d3f4633-maas-api-service-ca\") pod \"4ce94df4-03c6-4dad-aea6-b2462d3f4633\" (UID: \"4ce94df4-03c6-4dad-aea6-b2462d3f4633\") "
Jul 15 21:43:02.411438 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:02.411415    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ce94df4-03c6-4dad-aea6-b2462d3f4633-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ce94df4-03c6-4dad-aea6-b2462d3f4633" (UID: "4ce94df4-03c6-4dad-aea6-b2462d3f4633"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:43:02.512264 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:02.512179    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ce94df4-03c6-4dad-aea6-b2462d3f4633-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:43:03.197835 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:03.197805    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz"
Jul 15 21:43:03.198036 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:03.197797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-652862-qhxvz" event={"ID":"4ce94df4-03c6-4dad-aea6-b2462d3f4633","Type":"ContainerDied","Data":"31015ae0655b828db2935e199d138a37e0f94d50b9e0269964796399f43e95bb"}
Jul 15 21:43:03.198036 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:03.197897    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31015ae0655b828db2935e199d138a37e0f94d50b9e0269964796399f43e95bb"
Jul 15 21:43:10.478802 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.478757    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"]
Jul 15 21:43:10.479432 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.479033    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh" podUID="59d003f9-cb9b-4e79-982e-d26dbacefd05" containerName="maas-api" containerID="cri-o://a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d" gracePeriod=30
Jul 15 21:43:10.737815 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.737747    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:43:10.889326 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.889282    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4mk2\" (UniqueName: \"kubernetes.io/projected/59d003f9-cb9b-4e79-982e-d26dbacefd05-kube-api-access-v4mk2\") pod \"59d003f9-cb9b-4e79-982e-d26dbacefd05\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") "
Jul 15 21:43:10.889510 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.889474    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d003f9-cb9b-4e79-982e-d26dbacefd05-openshift-service-ca\") pod \"59d003f9-cb9b-4e79-982e-d26dbacefd05\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") "
Jul 15 21:43:10.889510 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.889508    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d003f9-cb9b-4e79-982e-d26dbacefd05-maas-api-tls\") pod \"59d003f9-cb9b-4e79-982e-d26dbacefd05\" (UID: \"59d003f9-cb9b-4e79-982e-d26dbacefd05\") "
Jul 15 21:43:10.889826 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.889802    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59d003f9-cb9b-4e79-982e-d26dbacefd05-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "59d003f9-cb9b-4e79-982e-d26dbacefd05" (UID: "59d003f9-cb9b-4e79-982e-d26dbacefd05"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:43:10.891454 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.891421    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d003f9-cb9b-4e79-982e-d26dbacefd05-kube-api-access-v4mk2" (OuterVolumeSpecName: "kube-api-access-v4mk2") pod "59d003f9-cb9b-4e79-982e-d26dbacefd05" (UID: "59d003f9-cb9b-4e79-982e-d26dbacefd05"). InnerVolumeSpecName "kube-api-access-v4mk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:43:10.891557 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.891527    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59d003f9-cb9b-4e79-982e-d26dbacefd05-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "59d003f9-cb9b-4e79-982e-d26dbacefd05" (UID: "59d003f9-cb9b-4e79-982e-d26dbacefd05"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:43:10.990368 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.990246    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4mk2\" (UniqueName: \"kubernetes.io/projected/59d003f9-cb9b-4e79-982e-d26dbacefd05-kube-api-access-v4mk2\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:43:10.990368 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.990291    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59d003f9-cb9b-4e79-982e-d26dbacefd05-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:43:10.990368 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:10.990306    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59d003f9-cb9b-4e79-982e-d26dbacefd05-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:43:11.255513 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:11.255423    2580 generic.go:356] "Generic (PLEG): container finished" podID="59d003f9-cb9b-4e79-982e-d26dbacefd05" containerID="a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d" exitCode=0
Jul 15 21:43:11.255513 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:11.255492    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"
Jul 15 21:43:11.255695 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:11.255517    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh" event={"ID":"59d003f9-cb9b-4e79-982e-d26dbacefd05","Type":"ContainerDied","Data":"a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d"}
Jul 15 21:43:11.255695 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:11.255578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh" event={"ID":"59d003f9-cb9b-4e79-982e-d26dbacefd05","Type":"ContainerDied","Data":"483d7986a78a0e6b10782af86ea1d24d13cebbd597ccccb301b835717fd734d0"}
Jul 15 21:43:11.255695 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:11.255608    2580 scope.go:117] "RemoveContainer" containerID="a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d"
Jul 15 21:43:11.264778 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:11.264754    2580 scope.go:117] "RemoveContainer" containerID="a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d"
Jul 15 21:43:11.265201 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:43:11.265181    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d\": container with ID starting with a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d not found: ID does not exist" containerID="a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d"
Jul 15 21:43:11.265272 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:11.265209    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d"} err="failed to get container status \"a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d\": rpc error: code = NotFound desc = could not find container \"a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d\": container with ID starting with a7bdb86e3cbe8087a7e9926cdb573994123c11aaa380d5abebae760caacb170d not found: ID does not exist"
Jul 15 21:43:11.276177 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:11.276144    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"]
Jul 15 21:43:11.282863 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:11.282832    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-652862-9dcb55c5d-4dsrh"]
Jul 15 21:43:13.135517 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:13.135479    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59d003f9-cb9b-4e79-982e-d26dbacefd05" path="/var/lib/kubelet/pods/59d003f9-cb9b-4e79-982e-d26dbacefd05/volumes"
Jul 15 21:43:25.688925 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.688884    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"]
Jul 15 21:43:25.689344 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.689286    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59d003f9-cb9b-4e79-982e-d26dbacefd05" containerName="maas-api"
Jul 15 21:43:25.689344 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.689296    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="59d003f9-cb9b-4e79-982e-d26dbacefd05" containerName="maas-api"
Jul 15 21:43:25.689344 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.689314    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ce94df4-03c6-4dad-aea6-b2462d3f4633" containerName="revoke-keys"
Jul 15 21:43:25.689344 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.689320    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ce94df4-03c6-4dad-aea6-b2462d3f4633" containerName="revoke-keys"
Jul 15 21:43:25.689474 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.689376    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ce94df4-03c6-4dad-aea6-b2462d3f4633" containerName="revoke-keys"
Jul 15 21:43:25.689474 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.689386    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="59d003f9-cb9b-4e79-982e-d26dbacefd05" containerName="maas-api"
Jul 15 21:43:25.692505 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.692488    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"
Jul 15 21:43:25.695132 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.695108    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:43:25.703109 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.703085    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"]
Jul 15 21:43:25.732735 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.732703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d00ada-d588-4372-a62e-201af656bd3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-da009d-l2hvk\" (UID: \"73d00ada-d588-4372-a62e-201af656bd3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"
Jul 15 21:43:25.834071 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.834029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d00ada-d588-4372-a62e-201af656bd3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-da009d-l2hvk\" (UID: \"73d00ada-d588-4372-a62e-201af656bd3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"
Jul 15 21:43:25.834691 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:25.834670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d00ada-d588-4372-a62e-201af656bd3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-da009d-l2hvk\" (UID: \"73d00ada-d588-4372-a62e-201af656bd3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"
Jul 15 21:43:26.004417 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:26.004335    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"
Jul 15 21:43:26.145801 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:26.145766    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"]
Jul 15 21:43:26.147392 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:43:26.147355    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d00ada_d588_4372_a62e_201af656bd3d.slice/crio-50e1c7b3f0ad1b7206f8cbe6d161d6e9a4d89138ea31cf040546bc852540e700 WatchSource:0}: Error finding container 50e1c7b3f0ad1b7206f8cbe6d161d6e9a4d89138ea31cf040546bc852540e700: Status 404 returned error can't find the container with id 50e1c7b3f0ad1b7206f8cbe6d161d6e9a4d89138ea31cf040546bc852540e700
Jul 15 21:43:26.341979 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:26.341927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk" event={"ID":"73d00ada-d588-4372-a62e-201af656bd3d","Type":"ContainerStarted","Data":"50e1c7b3f0ad1b7206f8cbe6d161d6e9a4d89138ea31cf040546bc852540e700"}
Jul 15 21:43:27.348981 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:27.348920    2580 generic.go:356] "Generic (PLEG): container finished" podID="73d00ada-d588-4372-a62e-201af656bd3d" containerID="515c2445dafe10d3784928d3477177c6ef7e29bdc169ee172ec5e6c0286bdd3c" exitCode=0
Jul 15 21:43:27.349457 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:27.348991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk" event={"ID":"73d00ada-d588-4372-a62e-201af656bd3d","Type":"ContainerDied","Data":"515c2445dafe10d3784928d3477177c6ef7e29bdc169ee172ec5e6c0286bdd3c"}
Jul 15 21:43:28.488302 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:28.488275    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"
Jul 15 21:43:28.560454 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:28.560412    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d00ada-d588-4372-a62e-201af656bd3d-maas-api-service-ca\") pod \"73d00ada-d588-4372-a62e-201af656bd3d\" (UID: \"73d00ada-d588-4372-a62e-201af656bd3d\") "
Jul 15 21:43:28.560793 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:28.560768    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73d00ada-d588-4372-a62e-201af656bd3d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73d00ada-d588-4372-a62e-201af656bd3d" (UID: "73d00ada-d588-4372-a62e-201af656bd3d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:43:28.662040 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:28.661929    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73d00ada-d588-4372-a62e-201af656bd3d-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:43:29.361109 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:29.361078    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk"
Jul 15 21:43:29.361277 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:29.361115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-da009d-l2hvk" event={"ID":"73d00ada-d588-4372-a62e-201af656bd3d","Type":"ContainerDied","Data":"50e1c7b3f0ad1b7206f8cbe6d161d6e9a4d89138ea31cf040546bc852540e700"}
Jul 15 21:43:29.361277 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:29.361170    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50e1c7b3f0ad1b7206f8cbe6d161d6e9a4d89138ea31cf040546bc852540e700"
Jul 15 21:43:36.163991 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.163941    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"]
Jul 15 21:43:36.165243 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.165188    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw" podUID="07b6e26a-81a4-43c5-bd48-6ff3002256f7" containerName="maas-api" containerID="cri-o://82538ff0ac2a7e128765fedc3eaf338803c797927343d4901c98b9bf1bbda4c6" gracePeriod=30
Jul 15 21:43:36.403318 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.403287    2580 generic.go:356] "Generic (PLEG): container finished" podID="07b6e26a-81a4-43c5-bd48-6ff3002256f7" containerID="82538ff0ac2a7e128765fedc3eaf338803c797927343d4901c98b9bf1bbda4c6" exitCode=0
Jul 15 21:43:36.403457 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.403361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw" event={"ID":"07b6e26a-81a4-43c5-bd48-6ff3002256f7","Type":"ContainerDied","Data":"82538ff0ac2a7e128765fedc3eaf338803c797927343d4901c98b9bf1bbda4c6"}
Jul 15 21:43:36.419245 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.419183    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:43:36.532568 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.532535    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7slh\" (UniqueName: \"kubernetes.io/projected/07b6e26a-81a4-43c5-bd48-6ff3002256f7-kube-api-access-q7slh\") pod \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") "
Jul 15 21:43:36.532734 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.532578    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b6e26a-81a4-43c5-bd48-6ff3002256f7-maas-api-tls\") pod \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") "
Jul 15 21:43:36.532734 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.532607    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b6e26a-81a4-43c5-bd48-6ff3002256f7-openshift-service-ca\") pod \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\" (UID: \"07b6e26a-81a4-43c5-bd48-6ff3002256f7\") "
Jul 15 21:43:36.533011 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.532980    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b6e26a-81a4-43c5-bd48-6ff3002256f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "07b6e26a-81a4-43c5-bd48-6ff3002256f7" (UID: "07b6e26a-81a4-43c5-bd48-6ff3002256f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:43:36.534789 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.534764    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07b6e26a-81a4-43c5-bd48-6ff3002256f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "07b6e26a-81a4-43c5-bd48-6ff3002256f7" (UID: "07b6e26a-81a4-43c5-bd48-6ff3002256f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:43:36.534846 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.534774    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b6e26a-81a4-43c5-bd48-6ff3002256f7-kube-api-access-q7slh" (OuterVolumeSpecName: "kube-api-access-q7slh") pod "07b6e26a-81a4-43c5-bd48-6ff3002256f7" (UID: "07b6e26a-81a4-43c5-bd48-6ff3002256f7"). InnerVolumeSpecName "kube-api-access-q7slh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:43:36.634035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.633987    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7slh\" (UniqueName: \"kubernetes.io/projected/07b6e26a-81a4-43c5-bd48-6ff3002256f7-kube-api-access-q7slh\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:43:36.634035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.634030    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07b6e26a-81a4-43c5-bd48-6ff3002256f7-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:43:36.634035 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:36.634042    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07b6e26a-81a4-43c5-bd48-6ff3002256f7-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:43:37.410854 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:37.410767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw" event={"ID":"07b6e26a-81a4-43c5-bd48-6ff3002256f7","Type":"ContainerDied","Data":"dfb919205192c4370417c848c32e53bfdc464188245a6c42e960a31e43c925f2"}
Jul 15 21:43:37.410854 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:37.410802    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"
Jul 15 21:43:37.410854 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:37.410831    2580 scope.go:117] "RemoveContainer" containerID="82538ff0ac2a7e128765fedc3eaf338803c797927343d4901c98b9bf1bbda4c6"
Jul 15 21:43:37.434671 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:37.434637    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"]
Jul 15 21:43:37.443991 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:37.443952    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-da009d-5fbf5485b8-k6vgw"]
Jul 15 21:43:39.135369 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:39.135327    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b6e26a-81a4-43c5-bd48-6ff3002256f7" path="/var/lib/kubelet/pods/07b6e26a-81a4-43c5-bd48-6ff3002256f7/volumes"
Jul 15 21:43:55.502648 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.502608    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"]
Jul 15 21:43:55.503206 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.503183    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73d00ada-d588-4372-a62e-201af656bd3d" containerName="revoke-keys"
Jul 15 21:43:55.503206 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.503207    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="73d00ada-d588-4372-a62e-201af656bd3d" containerName="revoke-keys"
Jul 15 21:43:55.503339 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.503228    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b6e26a-81a4-43c5-bd48-6ff3002256f7" containerName="maas-api"
Jul 15 21:43:55.503339 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.503234    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b6e26a-81a4-43c5-bd48-6ff3002256f7" containerName="maas-api"
Jul 15 21:43:55.503339 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.503314    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b6e26a-81a4-43c5-bd48-6ff3002256f7" containerName="maas-api"
Jul 15 21:43:55.503339 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.503324    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="73d00ada-d588-4372-a62e-201af656bd3d" containerName="revoke-keys"
Jul 15 21:43:55.506443 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.506424    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:55.508784 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.508757    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-31dfa2-serving-cert\""
Jul 15 21:43:55.508999 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.508953    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hxcvg\""
Jul 15 21:43:55.517105 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.517080    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"]
Jul 15 21:43:55.618191 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.618151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d262326e-7efc-478a-9a05-2b021aa6be2d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:55.618348 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.618206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d262326e-7efc-478a-9a05-2b021aa6be2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:55.618348 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.618262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcj8g\" (UniqueName: \"kubernetes.io/projected/d262326e-7efc-478a-9a05-2b021aa6be2d-kube-api-access-pcj8g\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:55.718872 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.718831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d262326e-7efc-478a-9a05-2b021aa6be2d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:55.719112 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.718892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d262326e-7efc-478a-9a05-2b021aa6be2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:55.719112 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:43:55.719033    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-serving-cert: secret "maas-api-e2e-shared-a-31dfa2-serving-cert" not found
Jul 15 21:43:55.719112 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.719067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcj8g\" (UniqueName: \"kubernetes.io/projected/d262326e-7efc-478a-9a05-2b021aa6be2d-kube-api-access-pcj8g\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:55.719112 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:43:55.719095    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d262326e-7efc-478a-9a05-2b021aa6be2d-maas-api-tls podName:d262326e-7efc-478a-9a05-2b021aa6be2d nodeName:}" failed. No retries permitted until 2026-07-15 21:43:56.219076565 +0000 UTC m=+2505.575995139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d262326e-7efc-478a-9a05-2b021aa6be2d-maas-api-tls") pod "maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m" (UID: "d262326e-7efc-478a-9a05-2b021aa6be2d") : secret "maas-api-e2e-shared-a-31dfa2-serving-cert" not found
Jul 15 21:43:55.719606 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.719586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d262326e-7efc-478a-9a05-2b021aa6be2d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:55.731437 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:55.731408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcj8g\" (UniqueName: \"kubernetes.io/projected/d262326e-7efc-478a-9a05-2b021aa6be2d-kube-api-access-pcj8g\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:56.223307 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:56.223263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d262326e-7efc-478a-9a05-2b021aa6be2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:56.225777 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:56.225743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d262326e-7efc-478a-9a05-2b021aa6be2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:56.419115 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:56.419077    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:56.563212 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:56.563188    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"]
Jul 15 21:43:56.565558 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:43:56.565527    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd262326e_7efc_478a_9a05_2b021aa6be2d.slice/crio-0ea1d49f3d6c877641571f60b90436531f3debd3c941ee31ec0d58d9dd4d82a8 WatchSource:0}: Error finding container 0ea1d49f3d6c877641571f60b90436531f3debd3c941ee31ec0d58d9dd4d82a8: Status 404 returned error can't find the container with id 0ea1d49f3d6c877641571f60b90436531f3debd3c941ee31ec0d58d9dd4d82a8
Jul 15 21:43:57.526110 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:57.526076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m" event={"ID":"d262326e-7efc-478a-9a05-2b021aa6be2d","Type":"ContainerStarted","Data":"a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022"}
Jul 15 21:43:57.526475 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:57.526129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m" event={"ID":"d262326e-7efc-478a-9a05-2b021aa6be2d","Type":"ContainerStarted","Data":"0ea1d49f3d6c877641571f60b90436531f3debd3c941ee31ec0d58d9dd4d82a8"}
Jul 15 21:43:57.526475 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:57.526187    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:43:57.545409 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:43:57.545352    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m" podStartSLOduration=2.143744405 podStartE2EDuration="2.54533346s" totalImagesPullingTime="401.589055ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:43:57.542699922 +0000 UTC m=+2506.899618521" watchObservedRunningTime="2026-07-15 21:43:57.54533346 +0000 UTC m=+2506.902252056"
Jul 15 21:44:03.540143 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.540056    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:44:03.626692 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.626659    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"]
Jul 15 21:44:03.630730 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.630703    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.633570 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.633547    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-b7f88c-serving-cert\""
Jul 15 21:44:03.649978 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.649937    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"]
Jul 15 21:44:03.693587 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.693550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqnwm\" (UniqueName: \"kubernetes.io/projected/ca767338-9c2a-4f48-917f-d84369b18b37-kube-api-access-rqnwm\") pod \"maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.693776 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.693659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca767338-9c2a-4f48-917f-d84369b18b37-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.693776 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.693696    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca767338-9c2a-4f48-917f-d84369b18b37-maas-api-tls\") pod \"maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.794371 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.794270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqnwm\" (UniqueName: \"kubernetes.io/projected/ca767338-9c2a-4f48-917f-d84369b18b37-kube-api-access-rqnwm\") pod \"maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.794371 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.794330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca767338-9c2a-4f48-917f-d84369b18b37-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.794371 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.794354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca767338-9c2a-4f48-917f-d84369b18b37-maas-api-tls\") pod \"maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.795115 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.795084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca767338-9c2a-4f48-917f-d84369b18b37-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.796735 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.796714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca767338-9c2a-4f48-917f-d84369b18b37-maas-api-tls\") pod \"maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.803061 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.803033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqnwm\" (UniqueName: \"kubernetes.io/projected/ca767338-9c2a-4f48-917f-d84369b18b37-kube-api-access-rqnwm\") pod \"maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:03.943771 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:03.943739    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:04.086814 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:04.086788    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"]
Jul 15 21:44:04.089110 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:44:04.089080    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca767338_9c2a_4f48_917f_d84369b18b37.slice/crio-06b0764327cd7168af50ef75f48c5a94519b490102554e6a28488232255be3f6 WatchSource:0}: Error finding container 06b0764327cd7168af50ef75f48c5a94519b490102554e6a28488232255be3f6: Status 404 returned error can't find the container with id 06b0764327cd7168af50ef75f48c5a94519b490102554e6a28488232255be3f6
Jul 15 21:44:04.565919 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:04.565886    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg" event={"ID":"ca767338-9c2a-4f48-917f-d84369b18b37","Type":"ContainerStarted","Data":"fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5"}
Jul 15 21:44:04.566347 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:04.565941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg" event={"ID":"ca767338-9c2a-4f48-917f-d84369b18b37","Type":"ContainerStarted","Data":"06b0764327cd7168af50ef75f48c5a94519b490102554e6a28488232255be3f6"}
Jul 15 21:44:04.566347 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:04.566086    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:44:04.591936 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:04.591875    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg" podStartSLOduration=1.229041158 podStartE2EDuration="1.591860016s" totalImagesPullingTime="362.818858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:44:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:44:04.590203405 +0000 UTC m=+2513.947122000" watchObservedRunningTime="2026-07-15 21:44:04.591860016 +0000 UTC m=+2513.948778633"
Jul 15 21:44:10.576970 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:44:10.576920    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:45:00.141772 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.141737    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p"]
Jul 15 21:45:00.145340 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.145324    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt"]
Jul 15 21:45:00.145458 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.145438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p"
Jul 15 21:45:00.148610 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.148593    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns"]
Jul 15 21:45:00.148730 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.148712    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt"
Jul 15 21:45:00.151881 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.151858    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:45:00.163950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.163682    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p"]
Jul 15 21:45:00.163950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.163809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns"
Jul 15 21:45:00.163950 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.163878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p"
Jul 15 21:45:00.168153 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.168015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt"]
Jul 15 21:45:00.171728 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.171710    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns"]
Jul 15 21:45:00.173593 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.173571    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt"
Jul 15 21:45:00.208908 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.208864    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns"
Jul 15 21:45:00.342019 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.341989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p"]
Jul 15 21:45:00.346249 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:45:00.346209    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff9e36d_6e67_4d26_951e_bd5e89e57bb5.slice/crio-3bbee46949f00b59a0b6b3ea3261c3e4ee51c254576508ab40f93d62df874782 WatchSource:0}: Error finding container 3bbee46949f00b59a0b6b3ea3261c3e4ee51c254576508ab40f93d62df874782: Status 404 returned error can't find the container with id 3bbee46949f00b59a0b6b3ea3261c3e4ee51c254576508ab40f93d62df874782
Jul 15 21:45:00.370182 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.370105    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt"]
Jul 15 21:45:00.372164 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:45:00.372124    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4c82391_6671_49fe_8e98_19754c262671.slice/crio-75a69341fcc8556bd61ebb982841c737e06a6235ee3fd03ed55a9d8057a8a05c WatchSource:0}: Error finding container 75a69341fcc8556bd61ebb982841c737e06a6235ee3fd03ed55a9d8057a8a05c: Status 404 returned error can't find the container with id 75a69341fcc8556bd61ebb982841c737e06a6235ee3fd03ed55a9d8057a8a05c
Jul 15 21:45:00.407456 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.407432    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns"]
Jul 15 21:45:00.411093 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:45:00.411058    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bf2948d_a7f6_42bf_9a93_4e37ffd71b16.slice/crio-0159119c27dfbfe4425295a089d4cbe0f28a8fa3fce2caa1f4f6d677c44029b5 WatchSource:0}: Error finding container 0159119c27dfbfe4425295a089d4cbe0f28a8fa3fce2caa1f4f6d677c44029b5: Status 404 returned error can't find the container with id 0159119c27dfbfe4425295a089d4cbe0f28a8fa3fce2caa1f4f6d677c44029b5
Jul 15 21:45:00.875564 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.875518    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p" event={"ID":"3ff9e36d-6e67-4d26-951e-bd5e89e57bb5","Type":"ContainerStarted","Data":"132322a6eb8e62c5159f06d3607f6e8d3038b9447119d078cea572ab28e0f9b2"}
Jul 15 21:45:00.875756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.875591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p" event={"ID":"3ff9e36d-6e67-4d26-951e-bd5e89e57bb5","Type":"ContainerStarted","Data":"3bbee46949f00b59a0b6b3ea3261c3e4ee51c254576508ab40f93d62df874782"}
Jul 15 21:45:00.877110 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.877084    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt" event={"ID":"c4c82391-6671-49fe-8e98-19754c262671","Type":"ContainerStarted","Data":"2fdd3f6122802a15775785916515a610fa971240400714db395d09cce0e054dd"}
Jul 15 21:45:00.877248 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.877137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt" event={"ID":"c4c82391-6671-49fe-8e98-19754c262671","Type":"ContainerStarted","Data":"75a69341fcc8556bd61ebb982841c737e06a6235ee3fd03ed55a9d8057a8a05c"}
Jul 15 21:45:00.878503 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.878479    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns" event={"ID":"4bf2948d-a7f6-42bf-9a93-4e37ffd71b16","Type":"ContainerStarted","Data":"544b780504f31cf333de8e0026cae6c6ebf439d8aa55ce843c5c25f1eedd5327"}
Jul 15 21:45:00.878627 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.878526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns" event={"ID":"4bf2948d-a7f6-42bf-9a93-4e37ffd71b16","Type":"ContainerStarted","Data":"0159119c27dfbfe4425295a089d4cbe0f28a8fa3fce2caa1f4f6d677c44029b5"}
Jul 15 21:45:00.891861 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.891816    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p" podStartSLOduration=0.891800903 podStartE2EDuration="891.800903ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:45:00.890442332 +0000 UTC m=+2570.247360929" watchObservedRunningTime="2026-07-15 21:45:00.891800903 +0000 UTC m=+2570.248719549"
Jul 15 21:45:00.906650 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.906550    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns" podStartSLOduration=0.906533988 podStartE2EDuration="906.533988ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:45:00.90410825 +0000 UTC m=+2570.261026848" watchObservedRunningTime="2026-07-15 21:45:00.906533988 +0000 UTC m=+2570.263452563"
Jul 15 21:45:00.920832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:00.920779    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt" podStartSLOduration=0.920762516 podStartE2EDuration="920.762516ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:45:00.919505471 +0000 UTC m=+2570.276424068" watchObservedRunningTime="2026-07-15 21:45:00.920762516 +0000 UTC m=+2570.277681113"
Jul 15 21:45:01.885909 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:01.885871    2580 generic.go:356] "Generic (PLEG): container finished" podID="3ff9e36d-6e67-4d26-951e-bd5e89e57bb5" containerID="132322a6eb8e62c5159f06d3607f6e8d3038b9447119d078cea572ab28e0f9b2" exitCode=0
Jul 15 21:45:01.886390 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:01.885950    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p" event={"ID":"3ff9e36d-6e67-4d26-951e-bd5e89e57bb5","Type":"ContainerDied","Data":"132322a6eb8e62c5159f06d3607f6e8d3038b9447119d078cea572ab28e0f9b2"}
Jul 15 21:45:01.887506 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:01.887476    2580 generic.go:356] "Generic (PLEG): container finished" podID="c4c82391-6671-49fe-8e98-19754c262671" containerID="2fdd3f6122802a15775785916515a610fa971240400714db395d09cce0e054dd" exitCode=0
Jul 15 21:45:01.887645 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:01.887554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt" event={"ID":"c4c82391-6671-49fe-8e98-19754c262671","Type":"ContainerDied","Data":"2fdd3f6122802a15775785916515a610fa971240400714db395d09cce0e054dd"}
Jul 15 21:45:01.888997 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:01.888972    2580 generic.go:356] "Generic (PLEG): container finished" podID="4bf2948d-a7f6-42bf-9a93-4e37ffd71b16" containerID="544b780504f31cf333de8e0026cae6c6ebf439d8aa55ce843c5c25f1eedd5327" exitCode=0
Jul 15 21:45:01.889131 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:01.889020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns" event={"ID":"4bf2948d-a7f6-42bf-9a93-4e37ffd71b16","Type":"ContainerDied","Data":"544b780504f31cf333de8e0026cae6c6ebf439d8aa55ce843c5c25f1eedd5327"}
Jul 15 21:45:03.047172 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.047092    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p"
Jul 15 21:45:03.072923 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.072896    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns"
Jul 15 21:45:03.091861 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.091831    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt"
Jul 15 21:45:03.901758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.901717    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns"
Jul 15 21:45:03.901758 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.901746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-lnmns" event={"ID":"4bf2948d-a7f6-42bf-9a93-4e37ffd71b16","Type":"ContainerDied","Data":"0159119c27dfbfe4425295a089d4cbe0f28a8fa3fce2caa1f4f6d677c44029b5"}
Jul 15 21:45:03.902023 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.901835    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0159119c27dfbfe4425295a089d4cbe0f28a8fa3fce2caa1f4f6d677c44029b5"
Jul 15 21:45:03.903163 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.903142    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p"
Jul 15 21:45:03.903293 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.903184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p" event={"ID":"3ff9e36d-6e67-4d26-951e-bd5e89e57bb5","Type":"ContainerDied","Data":"3bbee46949f00b59a0b6b3ea3261c3e4ee51c254576508ab40f93d62df874782"}
Jul 15 21:45:03.903293 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.903238    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bbee46949f00b59a0b6b3ea3261c3e4ee51c254576508ab40f93d62df874782"
Jul 15 21:45:03.904560 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.904536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt" event={"ID":"c4c82391-6671-49fe-8e98-19754c262671","Type":"ContainerDied","Data":"75a69341fcc8556bd61ebb982841c737e06a6235ee3fd03ed55a9d8057a8a05c"}
Jul 15 21:45:03.904560 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.904553    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt"
Jul 15 21:45:03.904686 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:45:03.904579    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75a69341fcc8556bd61ebb982841c737e06a6235ee3fd03ed55a9d8057a8a05c"
Jul 15 21:47:11.283514 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:11.283476    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:47:11.286734 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:11.286711    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:47:11.292856 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:11.292839    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:47:11.296124 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:11.296107    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gckpk_a5ff2fdc-8161-49f8-add9-8393e61dd94f/ovn-acl-logging/0.log"
Jul 15 21:47:26.581268 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581180    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"]
Jul 15 21:47:26.581756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581595    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c82391-6671-49fe-8e98-19754c262671" containerName="cleanup"
Jul 15 21:47:26.581756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581608    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c82391-6671-49fe-8e98-19754c262671" containerName="cleanup"
Jul 15 21:47:26.581756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581621    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff9e36d-6e67-4d26-951e-bd5e89e57bb5" containerName="cleanup"
Jul 15 21:47:26.581756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581627    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff9e36d-6e67-4d26-951e-bd5e89e57bb5" containerName="cleanup"
Jul 15 21:47:26.581756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581647    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bf2948d-a7f6-42bf-9a93-4e37ffd71b16" containerName="cleanup"
Jul 15 21:47:26.581756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581652    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bf2948d-a7f6-42bf-9a93-4e37ffd71b16" containerName="cleanup"
Jul 15 21:47:26.581756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581746    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4c82391-6671-49fe-8e98-19754c262671" containerName="cleanup"
Jul 15 21:47:26.581756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581757    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff9e36d-6e67-4d26-951e-bd5e89e57bb5" containerName="cleanup"
Jul 15 21:47:26.581756 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.581763    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bf2948d-a7f6-42bf-9a93-4e37ffd71b16" containerName="cleanup"
Jul 15 21:47:26.584911 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.584890    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.587977 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.587943    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-31dfa2\"/\"openshift-service-ca.crt\""
Jul 15 21:47:26.588084 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.588044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-31dfa2\"/\"sub-test-model-31dfa2-kserve-self-signed-certs\""
Jul 15 21:47:26.588406 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.588366    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-31dfa2\"/\"kube-root-ca.crt\""
Jul 15 21:47:26.588525 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.588503    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-31dfa2\"/\"default-dockercfg-822ck\""
Jul 15 21:47:26.594148 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.594126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"]
Jul 15 21:47:26.645040 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.645008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-tmp-dir\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.645040 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.645045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420d5efc-a614-4616-a8f1-1269de90b537-tls-certs\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.645262 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.645139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-dshm\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.645262 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.645223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdtqp\" (UniqueName: \"kubernetes.io/projected/420d5efc-a614-4616-a8f1-1269de90b537-kube-api-access-rdtqp\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.645332 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.645286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-home\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.645332 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.645315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-model-cache\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.746630 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.746591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-dshm\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.746834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.746754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdtqp\" (UniqueName: \"kubernetes.io/projected/420d5efc-a614-4616-a8f1-1269de90b537-kube-api-access-rdtqp\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.746834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.746788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-home\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.746834 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.746811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-model-cache\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.747076 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.747041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-tmp-dir\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.747169 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.747096    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420d5efc-a614-4616-a8f1-1269de90b537-tls-certs\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.747430 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.747406    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-model-cache\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.747430 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.747419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-home\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.747665 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.747642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-tmp-dir\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.749628 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.749606    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-dshm\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.750085 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.750067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420d5efc-a614-4616-a8f1-1269de90b537-tls-certs\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.762022 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.761996    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdtqp\" (UniqueName: \"kubernetes.io/projected/420d5efc-a614-4616-a8f1-1269de90b537-kube-api-access-rdtqp\") pod \"sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") " pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:26.896032 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:26.895926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:27.035888 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:27.035859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"]
Jul 15 21:47:27.037597 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:47:27.037572    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod420d5efc_a614_4616_a8f1_1269de90b537.slice/crio-3109c99b603e6db5a306de79015dcfc2b1837b42c69727ff73d160d736f3008f WatchSource:0}: Error finding container 3109c99b603e6db5a306de79015dcfc2b1837b42c69727ff73d160d736f3008f: Status 404 returned error can't find the container with id 3109c99b603e6db5a306de79015dcfc2b1837b42c69727ff73d160d736f3008f
Jul 15 21:47:27.689573 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:27.689537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx" event={"ID":"420d5efc-a614-4616-a8f1-1269de90b537","Type":"ContainerStarted","Data":"6c0a927a3731c9b88482525d36b69369b2912b698d336e17a60ef9a5a58c8030"}
Jul 15 21:47:27.690007 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:27.689586    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx" event={"ID":"420d5efc-a614-4616-a8f1-1269de90b537","Type":"ContainerStarted","Data":"3109c99b603e6db5a306de79015dcfc2b1837b42c69727ff73d160d736f3008f"}
Jul 15 21:47:27.708575 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:27.708523    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx" podStartSLOduration=1.7085092400000002 podStartE2EDuration="1.70850924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:47:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:47:27.707703446 +0000 UTC m=+2717.064622045" watchObservedRunningTime="2026-07-15 21:47:27.70850924 +0000 UTC m=+2717.065427836"
Jul 15 21:47:36.896124 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:36.896081    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:36.896510 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:36.896141    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:36.908195 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:36.908170    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:47:37.756616 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:47:37.756585    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:48:48.012908 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:48:48.012858    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"]
Jul 15 21:49:03.033788 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.033720    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx" podUID="420d5efc-a614-4616-a8f1-1269de90b537" containerName="main" containerID="cri-o://6c0a927a3731c9b88482525d36b69369b2912b698d336e17a60ef9a5a58c8030" gracePeriod=45
Jul 15 21:49:03.226830 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.226765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx" event={"ID":"420d5efc-a614-4616-a8f1-1269de90b537","Type":"ContainerDied","Data":"6c0a927a3731c9b88482525d36b69369b2912b698d336e17a60ef9a5a58c8030"}
Jul 15 21:49:03.226830 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.226715    2580 generic.go:356] "Generic (PLEG): container finished" podID="420d5efc-a614-4616-a8f1-1269de90b537" containerID="6c0a927a3731c9b88482525d36b69369b2912b698d336e17a60ef9a5a58c8030" exitCode=0
Jul 15 21:49:03.289527 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.289475    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:49:03.402376 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402354    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-model-cache\") pod \"420d5efc-a614-4616-a8f1-1269de90b537\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") "
Jul 15 21:49:03.402507 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402409    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-tmp-dir\") pod \"420d5efc-a614-4616-a8f1-1269de90b537\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") "
Jul 15 21:49:03.402507 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402431    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-home\") pod \"420d5efc-a614-4616-a8f1-1269de90b537\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") "
Jul 15 21:49:03.402507 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-dshm\") pod \"420d5efc-a614-4616-a8f1-1269de90b537\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") "
Jul 15 21:49:03.402686 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402540    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420d5efc-a614-4616-a8f1-1269de90b537-tls-certs\") pod \"420d5efc-a614-4616-a8f1-1269de90b537\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") "
Jul 15 21:49:03.402686 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402588    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdtqp\" (UniqueName: \"kubernetes.io/projected/420d5efc-a614-4616-a8f1-1269de90b537-kube-api-access-rdtqp\") pod \"420d5efc-a614-4616-a8f1-1269de90b537\" (UID: \"420d5efc-a614-4616-a8f1-1269de90b537\") "
Jul 15 21:49:03.402686 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402641    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-model-cache" (OuterVolumeSpecName: "model-cache") pod "420d5efc-a614-4616-a8f1-1269de90b537" (UID: "420d5efc-a614-4616-a8f1-1269de90b537"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:49:03.402850 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402717    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-home" (OuterVolumeSpecName: "home") pod "420d5efc-a614-4616-a8f1-1269de90b537" (UID: "420d5efc-a614-4616-a8f1-1269de90b537"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:49:03.402850 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402745    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "420d5efc-a614-4616-a8f1-1269de90b537" (UID: "420d5efc-a614-4616-a8f1-1269de90b537"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:49:03.402982 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402875    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-model-cache\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:49:03.402982 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402893    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-tmp-dir\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:49:03.402982 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.402902    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-home\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:49:03.404632 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.404610    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-dshm" (OuterVolumeSpecName: "dshm") pod "420d5efc-a614-4616-a8f1-1269de90b537" (UID: "420d5efc-a614-4616-a8f1-1269de90b537"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 21:49:03.404717 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.404698    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/420d5efc-a614-4616-a8f1-1269de90b537-kube-api-access-rdtqp" (OuterVolumeSpecName: "kube-api-access-rdtqp") pod "420d5efc-a614-4616-a8f1-1269de90b537" (UID: "420d5efc-a614-4616-a8f1-1269de90b537"). InnerVolumeSpecName "kube-api-access-rdtqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:49:03.404785 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.404763    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/420d5efc-a614-4616-a8f1-1269de90b537-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "420d5efc-a614-4616-a8f1-1269de90b537" (UID: "420d5efc-a614-4616-a8f1-1269de90b537"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:49:03.503544 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.503521    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420d5efc-a614-4616-a8f1-1269de90b537-dshm\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:49:03.503544 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.503542    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420d5efc-a614-4616-a8f1-1269de90b537-tls-certs\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:49:03.503674 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:03.503554    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdtqp\" (UniqueName: \"kubernetes.io/projected/420d5efc-a614-4616-a8f1-1269de90b537-kube-api-access-rdtqp\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:49:04.233484 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:04.233452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx" event={"ID":"420d5efc-a614-4616-a8f1-1269de90b537","Type":"ContainerDied","Data":"3109c99b603e6db5a306de79015dcfc2b1837b42c69727ff73d160d736f3008f"}
Jul 15 21:49:04.233484 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:04.233483    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"
Jul 15 21:49:04.233987 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:04.233501    2580 scope.go:117] "RemoveContainer" containerID="6c0a927a3731c9b88482525d36b69369b2912b698d336e17a60ef9a5a58c8030"
Jul 15 21:49:04.254814 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:04.254786    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"]
Jul 15 21:49:04.257195 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:04.257172    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-31dfa2/sub-test-model-31dfa2-kserve-6dd8cb7bcc-6zqtx"]
Jul 15 21:49:05.134354 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:49:05.134320    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="420d5efc-a614-4616-a8f1-1269de90b537" path="/var/lib/kubelet/pods/420d5efc-a614-4616-a8f1-1269de90b537/volumes"
Jul 15 21:50:24.953714 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:24.953632    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"]
Jul 15 21:50:24.954203 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:24.954080    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="420d5efc-a614-4616-a8f1-1269de90b537" containerName="main"
Jul 15 21:50:24.954203 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:24.954093    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="420d5efc-a614-4616-a8f1-1269de90b537" containerName="main"
Jul 15 21:50:24.954203 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:24.954167    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="420d5efc-a614-4616-a8f1-1269de90b537" containerName="main"
Jul 15 21:50:24.957266 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:24.957250    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"
Jul 15 21:50:24.959686 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:24.959662    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:50:24.963152 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:24.963125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"]
Jul 15 21:50:24.998561 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:24.998515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2842006f-4ffb-49c1-8b8f-e081c2b86809-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv\" (UID: \"2842006f-4ffb-49c1-8b8f-e081c2b86809\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"
Jul 15 21:50:25.099821 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:25.099786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2842006f-4ffb-49c1-8b8f-e081c2b86809-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv\" (UID: \"2842006f-4ffb-49c1-8b8f-e081c2b86809\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"
Jul 15 21:50:25.100468 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:25.100448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2842006f-4ffb-49c1-8b8f-e081c2b86809-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv\" (UID: \"2842006f-4ffb-49c1-8b8f-e081c2b86809\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"
Jul 15 21:50:25.270761 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:25.270685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"
Jul 15 21:50:25.411375 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:25.411343    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"]
Jul 15 21:50:25.673459 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:25.673423    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv" event={"ID":"2842006f-4ffb-49c1-8b8f-e081c2b86809","Type":"ContainerStarted","Data":"726b144a3a6a0c161ae213a26cc8946493187ffe279ec46be28e155f5f3435b0"}
Jul 15 21:50:25.673605 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:25.673480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv" event={"ID":"2842006f-4ffb-49c1-8b8f-e081c2b86809","Type":"ContainerStarted","Data":"55ff44c7acc4a16fda52aa5abd4a6eb53cf26e7a01af7fb4114e781df336d0c7"}
Jul 15 21:50:25.689550 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:25.689493    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv" podStartSLOduration=1.689479475 podStartE2EDuration="1.689479475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:50:25.687554275 +0000 UTC m=+2895.044472872" watchObservedRunningTime="2026-07-15 21:50:25.689479475 +0000 UTC m=+2895.046398118"
Jul 15 21:50:26.681103 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:26.681064    2580 generic.go:356] "Generic (PLEG): container finished" podID="2842006f-4ffb-49c1-8b8f-e081c2b86809" containerID="726b144a3a6a0c161ae213a26cc8946493187ffe279ec46be28e155f5f3435b0" exitCode=0
Jul 15 21:50:26.681491 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:26.681147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv" event={"ID":"2842006f-4ffb-49c1-8b8f-e081c2b86809","Type":"ContainerDied","Data":"726b144a3a6a0c161ae213a26cc8946493187ffe279ec46be28e155f5f3435b0"}
Jul 15 21:50:27.817352 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:27.817325    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"
Jul 15 21:50:27.925395 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:27.925361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2842006f-4ffb-49c1-8b8f-e081c2b86809-maas-api-service-ca\") pod \"2842006f-4ffb-49c1-8b8f-e081c2b86809\" (UID: \"2842006f-4ffb-49c1-8b8f-e081c2b86809\") "
Jul 15 21:50:27.925740 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:27.925717    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2842006f-4ffb-49c1-8b8f-e081c2b86809-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2842006f-4ffb-49c1-8b8f-e081c2b86809" (UID: "2842006f-4ffb-49c1-8b8f-e081c2b86809"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:50:28.027009 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:28.026901    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2842006f-4ffb-49c1-8b8f-e081c2b86809-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:50:28.692554 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:28.692519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv" event={"ID":"2842006f-4ffb-49c1-8b8f-e081c2b86809","Type":"ContainerDied","Data":"55ff44c7acc4a16fda52aa5abd4a6eb53cf26e7a01af7fb4114e781df336d0c7"}
Jul 15 21:50:28.692733 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:28.692571    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55ff44c7acc4a16fda52aa5abd4a6eb53cf26e7a01af7fb4114e781df336d0c7"
Jul 15 21:50:28.692733 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:28.692538    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv"
Jul 15 21:50:35.396733 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.396692    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"]
Jul 15 21:50:35.397306 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.397020    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m" podUID="d262326e-7efc-478a-9a05-2b021aa6be2d" containerName="maas-api" containerID="cri-o://a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022" gracePeriod=30
Jul 15 21:50:35.426931 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.426898    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p"]
Jul 15 21:50:35.429809 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.429782    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-31dfa2-29735865-7qw5p"]
Jul 15 21:50:35.654846 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.654784    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:50:35.732467 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.732432    2580 generic.go:356] "Generic (PLEG): container finished" podID="d262326e-7efc-478a-9a05-2b021aa6be2d" containerID="a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022" exitCode=0
Jul 15 21:50:35.732669 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.732477    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"
Jul 15 21:50:35.732669 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.732516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m" event={"ID":"d262326e-7efc-478a-9a05-2b021aa6be2d","Type":"ContainerDied","Data":"a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022"}
Jul 15 21:50:35.732669 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.732575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m" event={"ID":"d262326e-7efc-478a-9a05-2b021aa6be2d","Type":"ContainerDied","Data":"0ea1d49f3d6c877641571f60b90436531f3debd3c941ee31ec0d58d9dd4d82a8"}
Jul 15 21:50:35.732669 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.732598    2580 scope.go:117] "RemoveContainer" containerID="a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022"
Jul 15 21:50:35.741313 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.741296    2580 scope.go:117] "RemoveContainer" containerID="a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022"
Jul 15 21:50:35.741568 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:50:35.741547    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022\": container with ID starting with a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022 not found: ID does not exist" containerID="a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022"
Jul 15 21:50:35.741638 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.741580    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022"} err="failed to get container status \"a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022\": rpc error: code = NotFound desc = could not find container \"a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022\": container with ID starting with a85a248e33fbdaf1de2bcaaf759262c62598e8be4808a5873ae1c93a8897d022 not found: ID does not exist"
Jul 15 21:50:35.802223 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.802187    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d262326e-7efc-478a-9a05-2b021aa6be2d-openshift-service-ca\") pod \"d262326e-7efc-478a-9a05-2b021aa6be2d\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") "
Jul 15 21:50:35.802370 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.802239    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d262326e-7efc-478a-9a05-2b021aa6be2d-maas-api-tls\") pod \"d262326e-7efc-478a-9a05-2b021aa6be2d\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") "
Jul 15 21:50:35.802370 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.802278    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcj8g\" (UniqueName: \"kubernetes.io/projected/d262326e-7efc-478a-9a05-2b021aa6be2d-kube-api-access-pcj8g\") pod \"d262326e-7efc-478a-9a05-2b021aa6be2d\" (UID: \"d262326e-7efc-478a-9a05-2b021aa6be2d\") "
Jul 15 21:50:35.802578 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.802556    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d262326e-7efc-478a-9a05-2b021aa6be2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d262326e-7efc-478a-9a05-2b021aa6be2d" (UID: "d262326e-7efc-478a-9a05-2b021aa6be2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:50:35.804402 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.804382    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d262326e-7efc-478a-9a05-2b021aa6be2d-kube-api-access-pcj8g" (OuterVolumeSpecName: "kube-api-access-pcj8g") pod "d262326e-7efc-478a-9a05-2b021aa6be2d" (UID: "d262326e-7efc-478a-9a05-2b021aa6be2d"). InnerVolumeSpecName "kube-api-access-pcj8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:50:35.804462 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.804383    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d262326e-7efc-478a-9a05-2b021aa6be2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d262326e-7efc-478a-9a05-2b021aa6be2d" (UID: "d262326e-7efc-478a-9a05-2b021aa6be2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:50:35.903726 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.903686    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d262326e-7efc-478a-9a05-2b021aa6be2d-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:50:35.903726 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.903720    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d262326e-7efc-478a-9a05-2b021aa6be2d-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:50:35.903726 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:35.903733    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcj8g\" (UniqueName: \"kubernetes.io/projected/d262326e-7efc-478a-9a05-2b021aa6be2d-kube-api-access-pcj8g\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:50:36.055109 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:36.055067    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"]
Jul 15 21:50:36.057379 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:36.057352    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-31dfa2-744c8b5c59-7sb7m"]
Jul 15 21:50:37.134930 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:37.134894    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ff9e36d-6e67-4d26-951e-bd5e89e57bb5" path="/var/lib/kubelet/pods/3ff9e36d-6e67-4d26-951e-bd5e89e57bb5/volumes"
Jul 15 21:50:37.135317 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:37.135265    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d262326e-7efc-478a-9a05-2b021aa6be2d" path="/var/lib/kubelet/pods/d262326e-7efc-478a-9a05-2b021aa6be2d/volumes"
Jul 15 21:50:51.124195 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.124152    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"]
Jul 15 21:50:51.124729 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.124708    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2842006f-4ffb-49c1-8b8f-e081c2b86809" containerName="revoke-keys"
Jul 15 21:50:51.124811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.124731    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2842006f-4ffb-49c1-8b8f-e081c2b86809" containerName="revoke-keys"
Jul 15 21:50:51.124811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.124757    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d262326e-7efc-478a-9a05-2b021aa6be2d" containerName="maas-api"
Jul 15 21:50:51.124811 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.124766    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d262326e-7efc-478a-9a05-2b021aa6be2d" containerName="maas-api"
Jul 15 21:50:51.124984 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.124860    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2842006f-4ffb-49c1-8b8f-e081c2b86809" containerName="revoke-keys"
Jul 15 21:50:51.124984 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.124872    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d262326e-7efc-478a-9a05-2b021aa6be2d" containerName="maas-api"
Jul 15 21:50:51.131216 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.131186    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"
Jul 15 21:50:51.136743 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.136715    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9k9jm\""
Jul 15 21:50:51.146974 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.146925    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"]
Jul 15 21:50:51.241826 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.241790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c7cae-8111-47af-964b-c6be91ff89ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq\" (UID: \"bf0c7cae-8111-47af-964b-c6be91ff89ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"
Jul 15 21:50:51.343319 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.343276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c7cae-8111-47af-964b-c6be91ff89ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq\" (UID: \"bf0c7cae-8111-47af-964b-c6be91ff89ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"
Jul 15 21:50:51.343908 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.343883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c7cae-8111-47af-964b-c6be91ff89ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq\" (UID: \"bf0c7cae-8111-47af-964b-c6be91ff89ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"
Jul 15 21:50:51.450324 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.450232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"
Jul 15 21:50:51.584581 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.584555    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"]
Jul 15 21:50:51.588240 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:50:51.588212    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0c7cae_8111_47af_964b_c6be91ff89ec.slice/crio-ad0cc623e8fa93f3b307b8290ceb5e532b53d118c4f08f33cdb7467edc61714a WatchSource:0}: Error finding container ad0cc623e8fa93f3b307b8290ceb5e532b53d118c4f08f33cdb7467edc61714a: Status 404 returned error can't find the container with id ad0cc623e8fa93f3b307b8290ceb5e532b53d118c4f08f33cdb7467edc61714a
Jul 15 21:50:51.819224 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.819181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq" event={"ID":"bf0c7cae-8111-47af-964b-c6be91ff89ec","Type":"ContainerStarted","Data":"259e1c7e04cb8dfccf45575046d2eceebcb7b4309fabbd44dc773e14f8f675ad"}
Jul 15 21:50:51.819472 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.819238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq" event={"ID":"bf0c7cae-8111-47af-964b-c6be91ff89ec","Type":"ContainerStarted","Data":"ad0cc623e8fa93f3b307b8290ceb5e532b53d118c4f08f33cdb7467edc61714a"}
Jul 15 21:50:51.837705 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:51.837651    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq" podStartSLOduration=0.837634735 podStartE2EDuration="837.634735ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:50:51.834542345 +0000 UTC m=+2921.191460932" watchObservedRunningTime="2026-07-15 21:50:51.837634735 +0000 UTC m=+2921.194553331"
Jul 15 21:50:52.826575 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:52.826541    2580 generic.go:356] "Generic (PLEG): container finished" podID="bf0c7cae-8111-47af-964b-c6be91ff89ec" containerID="259e1c7e04cb8dfccf45575046d2eceebcb7b4309fabbd44dc773e14f8f675ad" exitCode=0
Jul 15 21:50:52.826998 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:52.826631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq" event={"ID":"bf0c7cae-8111-47af-964b-c6be91ff89ec","Type":"ContainerDied","Data":"259e1c7e04cb8dfccf45575046d2eceebcb7b4309fabbd44dc773e14f8f675ad"}
Jul 15 21:50:53.965296 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:53.965269    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"
Jul 15 21:50:53.968403 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:53.968385    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c7cae-8111-47af-964b-c6be91ff89ec-maas-api-service-ca\") pod \"bf0c7cae-8111-47af-964b-c6be91ff89ec\" (UID: \"bf0c7cae-8111-47af-964b-c6be91ff89ec\") "
Jul 15 21:50:53.968699 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:53.968680    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0c7cae-8111-47af-964b-c6be91ff89ec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf0c7cae-8111-47af-964b-c6be91ff89ec" (UID: "bf0c7cae-8111-47af-964b-c6be91ff89ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:50:54.068953 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:54.068918    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c7cae-8111-47af-964b-c6be91ff89ec-maas-api-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:50:54.838862 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:54.838830    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq"
Jul 15 21:50:54.839064 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:54.838860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq" event={"ID":"bf0c7cae-8111-47af-964b-c6be91ff89ec","Type":"ContainerDied","Data":"ad0cc623e8fa93f3b307b8290ceb5e532b53d118c4f08f33cdb7467edc61714a"}
Jul 15 21:50:54.839064 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:50:54.838908    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad0cc623e8fa93f3b307b8290ceb5e532b53d118c4f08f33cdb7467edc61714a"
Jul 15 21:51:01.570803 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.570769    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"]
Jul 15 21:51:01.571238 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.571059    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg" podUID="ca767338-9c2a-4f48-917f-d84369b18b37" containerName="maas-api" containerID="cri-o://fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5" gracePeriod=30
Jul 15 21:51:01.596296 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.596263    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt"]
Jul 15 21:51:01.605357 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.599568    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-b7f88c-29735865-489dt"]
Jul 15 21:51:01.820295 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.820269    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:51:01.836648 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.836178    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca767338-9c2a-4f48-917f-d84369b18b37-maas-api-tls\") pod \"ca767338-9c2a-4f48-917f-d84369b18b37\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") "
Jul 15 21:51:01.836648 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.836312    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqnwm\" (UniqueName: \"kubernetes.io/projected/ca767338-9c2a-4f48-917f-d84369b18b37-kube-api-access-rqnwm\") pod \"ca767338-9c2a-4f48-917f-d84369b18b37\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") "
Jul 15 21:51:01.836648 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.836380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca767338-9c2a-4f48-917f-d84369b18b37-openshift-service-ca\") pod \"ca767338-9c2a-4f48-917f-d84369b18b37\" (UID: \"ca767338-9c2a-4f48-917f-d84369b18b37\") "
Jul 15 21:51:01.836949 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.836918    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca767338-9c2a-4f48-917f-d84369b18b37-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca767338-9c2a-4f48-917f-d84369b18b37" (UID: "ca767338-9c2a-4f48-917f-d84369b18b37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 21:51:01.839560 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.839511    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca767338-9c2a-4f48-917f-d84369b18b37-kube-api-access-rqnwm" (OuterVolumeSpecName: "kube-api-access-rqnwm") pod "ca767338-9c2a-4f48-917f-d84369b18b37" (UID: "ca767338-9c2a-4f48-917f-d84369b18b37"). InnerVolumeSpecName "kube-api-access-rqnwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 21:51:01.839560 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.839520    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca767338-9c2a-4f48-917f-d84369b18b37-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca767338-9c2a-4f48-917f-d84369b18b37" (UID: "ca767338-9c2a-4f48-917f-d84369b18b37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 21:51:01.884297 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.884248    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca767338-9c2a-4f48-917f-d84369b18b37" containerID="fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5" exitCode=0
Jul 15 21:51:01.884297 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.884297    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"
Jul 15 21:51:01.884494 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.884287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg" event={"ID":"ca767338-9c2a-4f48-917f-d84369b18b37","Type":"ContainerDied","Data":"fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5"}
Jul 15 21:51:01.884494 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.884436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg" event={"ID":"ca767338-9c2a-4f48-917f-d84369b18b37","Type":"ContainerDied","Data":"06b0764327cd7168af50ef75f48c5a94519b490102554e6a28488232255be3f6"}
Jul 15 21:51:01.884494 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.884462    2580 scope.go:117] "RemoveContainer" containerID="fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5"
Jul 15 21:51:01.895589 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.895546    2580 scope.go:117] "RemoveContainer" containerID="fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5"
Jul 15 21:51:01.896008 ip-10-0-130-29 kubenswrapper[2580]: E0715 21:51:01.895982    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5\": container with ID starting with fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5 not found: ID does not exist" containerID="fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5"
Jul 15 21:51:01.896097 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.896018    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5"} err="failed to get container status \"fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5\": rpc error: code = NotFound desc = could not find container \"fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5\": container with ID starting with fe1a25c413d61c9ec84bdbfa574fbc814c57837d187d54dab12b44d1237f5eb5 not found: ID does not exist"
Jul 15 21:51:01.905380 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.905358    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"]
Jul 15 21:51:01.909568 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.909543    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b7f88c-644dbbb49-9q2xg"]
Jul 15 21:51:01.937580 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.937547    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca767338-9c2a-4f48-917f-d84369b18b37-openshift-service-ca\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:51:01.937580 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.937577    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca767338-9c2a-4f48-917f-d84369b18b37-maas-api-tls\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:51:01.937727 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:01.937592    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqnwm\" (UniqueName: \"kubernetes.io/projected/ca767338-9c2a-4f48-917f-d84369b18b37-kube-api-access-rqnwm\") on node \"ip-10-0-130-29.ec2.internal\" DevicePath \"\""
Jul 15 21:51:03.134740 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:03.134699    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4c82391-6671-49fe-8e98-19754c262671" path="/var/lib/kubelet/pods/c4c82391-6671-49fe-8e98-19754c262671/volumes"
Jul 15 21:51:03.135279 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:03.135261    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca767338-9c2a-4f48-917f-d84369b18b37" path="/var/lib/kubelet/pods/ca767338-9c2a-4f48-917f-d84369b18b37/volumes"
Jul 15 21:51:11.483326 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:11.483289    2580 scope.go:117] "RemoveContainer" containerID="2fdd3f6122802a15775785916515a610fa971240400714db395d09cce0e054dd"
Jul 15 21:51:11.491605 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:11.491586    2580 scope.go:117] "RemoveContainer" containerID="132322a6eb8e62c5159f06d3607f6e8d3038b9447119d078cea572ab28e0f9b2"
Jul 15 21:51:24.365310 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:24.365277    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b45b4f795-9kmmd_8291a711-84ae-42ff-ba75-02528d6525de/manager/1.log"
Jul 15 21:51:24.516314 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:24.516286    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-tqlcz_9357ba15-5356-44b6-9451-d10b7c9f0616/server/0.log"
Jul 15 21:51:24.921832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:24.921804    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c67f598f-qjkkv_e37669e5-eb85-4e79-a7bc-83a947ca45b6/copy-manifests/0.log"
Jul 15 21:51:24.934009 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:24.933978    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c67f598f-qjkkv_e37669e5-eb85-4e79-a7bc-83a947ca45b6/manager/0.log"
Jul 15 21:51:25.512410 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:25.512380    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5fd78dcc94-ltqxg_82da60cd-987f-4823-80ff-8460151853ca/maas-api/0.log"
Jul 15 21:51:25.642383 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:25.642355    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735835-mtbvv_7b899e9d-5d54-40b1-b104-dbd74ba0166b/cleanup/0.log"
Jul 15 21:51:25.766268 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:25.766193    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735850-6cwtm_74e395eb-f560-4445-9c11-b2b562a9f8f7/cleanup/0.log"
Jul 15 21:51:25.905226 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:25.905200    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735865-lnmns_4bf2948d-a7f6-42bf-9a93-4e37ffd71b16/cleanup/0.log"
Jul 15 21:51:26.048276 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:26.048252    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-8cee6bf5-lgnq9_ade0dc96-519d-444e-bfbe-fcf196e9b568/revoke-keys/0.log"
Jul 15 21:51:26.171921 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:26.171890    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-b0fcdc21-x78r7_bf94615b-3631-4f31-a464-80b7727bbb40/revoke-keys/0.log"
Jul 15 21:51:26.297336 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:26.297308    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-652862-qhxvz_4ce94df4-03c6-4dad-aea6-b2462d3f4633/revoke-keys/0.log"
Jul 15 21:51:26.435721 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:26.435633    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-da009d-l2hvk_73d00ada-d588-4372-a62e-201af656bd3d/revoke-keys/0.log"
Jul 15 21:51:26.570259 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:26.570233    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-994ab52c-9qrpn_c4675bd5-e82f-4443-8f45-3aa2b6e78159/revoke-keys/0.log"
Jul 15 21:51:26.700385 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:26.700307    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-7c4663-pccpr_0e7b4fd3-fe47-445c-9e14-98b4291b22d9/revoke-keys/0.log"
Jul 15 21:51:26.822919 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:26.822891    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-c419c9-dqfv5_60dbd51a-0086-4764-8781-98fcdb340e51/revoke-keys/0.log"
Jul 15 21:51:26.949817 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:26.949786    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-18bc3701-pxp5l_27d4f993-2abc-432d-aaed-59e85461ea79/revoke-keys/0.log"
Jul 15 21:51:27.075864 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:27.075835    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-0af6f2e9-bfzv4_57e80285-1cf6-4f6b-b20a-4de4063591c0/revoke-keys/0.log"
Jul 15 21:51:27.204090 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:27.204057    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-42fd7c24-g74h7_a4ed8bdb-b9c3-4157-a85b-4051793131e7/revoke-keys/0.log"
Jul 15 21:51:27.355743 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:27.355661    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-31dfa2-2qrcv_2842006f-4ffb-49c1-8b8f-e081c2b86809/revoke-keys/0.log"
Jul 15 21:51:27.483635 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:27.483604    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-b7f88c-xldwq_bf0c7cae-8111-47af-964b-c6be91ff89ec/revoke-keys/0.log"
Jul 15 21:51:30.636911 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:30.636880    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xvvsh_dee496e1-3e51-4ff6-85fb-9058fcf1d770/discovery/0.log"
Jul 15 21:51:31.166911 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:31.166883    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-4lwtr_e3e3752e-0960-4f15-be87-6e1ee6c81472/payload-pre-processing/0.log"
Jul 15 21:51:31.305826 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:31.305794    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-bpf2x_16d252fd-b5c2-4179-9bdd-cde357c9d355/payload-processing/0.log"
Jul 15 21:51:32.126334 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:32.126247    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vlf8x_7421969e-1676-4071-9336-0c601e1eb2fa/main/0.log"
Jul 15 21:51:32.407207 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:32.407132    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-xg8tp_ffb403a4-2722-48e4-afb2-52f9e2e75480/main/0.log"
Jul 15 21:51:42.067820 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:42.067788    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6ldsz_4e184053-7574-434b-8873-db1b343c9589/global-pull-secret-syncer/0.log"
Jul 15 21:51:42.198814 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:42.198787    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nhsnr_3517b0c1-318b-457b-b272-efc0e289dbbc/konnectivity-agent/0.log"
Jul 15 21:51:42.254416 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:42.254385    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-29.ec2.internal_46ea3c93a970f7700a2a3849415d7a22/haproxy/0.log"
Jul 15 21:51:50.427552 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:50.427516    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7gk99_254e79cf-8922-4bf5-a354-5f08af660818/kube-state-metrics/0.log"
Jul 15 21:51:50.448889 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:50.448858    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7gk99_254e79cf-8922-4bf5-a354-5f08af660818/kube-rbac-proxy-main/0.log"
Jul 15 21:51:50.486884 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:50.486858    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7gk99_254e79cf-8922-4bf5-a354-5f08af660818/kube-rbac-proxy-self/0.log"
Jul 15 21:51:50.517147 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:50.517125    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6f78695788-j9vmj_adff73c0-7575-47dd-a9cf-6323394c14fa/metrics-server/0.log"
Jul 15 21:51:50.588164 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:50.588132    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4z7rk_45041402-dc9b-404d-a815-99e68f9f1e21/node-exporter/0.log"
Jul 15 21:51:50.617258 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:50.617232    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4z7rk_45041402-dc9b-404d-a815-99e68f9f1e21/kube-rbac-proxy/0.log"
Jul 15 21:51:50.643031 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:50.643003    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4z7rk_45041402-dc9b-404d-a815-99e68f9f1e21/init-textfile/0.log"
Jul 15 21:51:51.287324 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:51.287293    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b5d98d4-hrzlc_74bce824-176c-460a-bcb5-362067d306f4/thanos-query/0.log"
Jul 15 21:51:51.308378 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:51.308332    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b5d98d4-hrzlc_74bce824-176c-460a-bcb5-362067d306f4/kube-rbac-proxy-web/0.log"
Jul 15 21:51:51.330153 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:51.330121    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b5d98d4-hrzlc_74bce824-176c-460a-bcb5-362067d306f4/kube-rbac-proxy/0.log"
Jul 15 21:51:51.351250 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:51.351219    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b5d98d4-hrzlc_74bce824-176c-460a-bcb5-362067d306f4/prom-label-proxy/0.log"
Jul 15 21:51:51.375366 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:51.375333    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b5d98d4-hrzlc_74bce824-176c-460a-bcb5-362067d306f4/kube-rbac-proxy-rules/0.log"
Jul 15 21:51:51.404153 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:51.404128    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b5d98d4-hrzlc_74bce824-176c-460a-bcb5-362067d306f4/kube-rbac-proxy-metrics/0.log"
Jul 15 21:51:52.366157 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.366117    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"]
Jul 15 21:51:52.366770 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.366705    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf0c7cae-8111-47af-964b-c6be91ff89ec" containerName="revoke-keys"
Jul 15 21:51:52.366770 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.366722    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf0c7cae-8111-47af-964b-c6be91ff89ec" containerName="revoke-keys"
Jul 15 21:51:52.366770 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.366743    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca767338-9c2a-4f48-917f-d84369b18b37" containerName="maas-api"
Jul 15 21:51:52.366770 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.366748    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca767338-9c2a-4f48-917f-d84369b18b37" containerName="maas-api"
Jul 15 21:51:52.366924 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.366820    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca767338-9c2a-4f48-917f-d84369b18b37" containerName="maas-api"
Jul 15 21:51:52.366924 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.366835    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf0c7cae-8111-47af-964b-c6be91ff89ec" containerName="revoke-keys"
Jul 15 21:51:52.370133 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.370116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.372204 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.372182    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqvxt\"/\"kube-root-ca.crt\""
Jul 15 21:51:52.372308 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.372206    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pqvxt\"/\"default-dockercfg-c2jrj\""
Jul 15 21:51:52.373029 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.372999    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqvxt\"/\"openshift-service-ca.crt\""
Jul 15 21:51:52.376459 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.376436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-sys\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.377249 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.376595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-lib-modules\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.377316 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.377263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-podres\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.377359 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.377321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-proc\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.377359 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.377348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sm6k\" (UniqueName: \"kubernetes.io/projected/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-kube-api-access-8sm6k\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.379623 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.379601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"]
Jul 15 21:51:52.478787 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.478748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-proc\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.478787 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.478785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sm6k\" (UniqueName: \"kubernetes.io/projected/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-kube-api-access-8sm6k\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.479067 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.478814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-sys\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.479067 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.478844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-lib-modules\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.479067 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.478874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-proc\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.479067 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.478902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-sys\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.479067 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.478990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-podres\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.479067 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.479042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-lib-modules\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.479304 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.479110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-podres\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.487331 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.487310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sm6k\" (UniqueName: \"kubernetes.io/projected/b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f-kube-api-access-8sm6k\") pod \"perf-node-gather-daemonset-d7tjr\" (UID: \"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f\") " pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.584262 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.584230    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-tq7ds_02992d82-53fc-4b51-9da7-0f59ecdce970/networking-console-plugin/0.log"
Jul 15 21:51:52.681359 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.681254    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:52.817875 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:52.817846    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"]
Jul 15 21:51:52.819198 ip-10-0-130-29 kubenswrapper[2580]: W0715 21:51:52.819171    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb86d4ee0_c7ec_4a47_b2cf_9cb2bcdcf15f.slice/crio-c551210c83756eb1858b19b5774ff65a4e20dabbbbe5a5c4473e416c58ab4ce4 WatchSource:0}: Error finding container c551210c83756eb1858b19b5774ff65a4e20dabbbbe5a5c4473e416c58ab4ce4: Status 404 returned error can't find the container with id c551210c83756eb1858b19b5774ff65a4e20dabbbbe5a5c4473e416c58ab4ce4
Jul 15 21:51:53.159867 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:53.159830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr" event={"ID":"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f","Type":"ContainerStarted","Data":"eb48f4087a5dab1fa89c6604f17f929c13d72fc48c6ab29cbc960dc35add28fc"}
Jul 15 21:51:53.160074 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:53.159884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr" event={"ID":"b86d4ee0-c7ec-4a47-b2cf-9cb2bcdcf15f","Type":"ContainerStarted","Data":"c551210c83756eb1858b19b5774ff65a4e20dabbbbe5a5c4473e416c58ab4ce4"}
Jul 15 21:51:53.160074 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:53.159949    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:51:53.163157 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:53.163128    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/2.log"
Jul 15 21:51:53.167248 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:53.167226    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-mjds2_72014806-cd64-45ba-be82-a71c7d7076fb/console-operator/3.log"
Jul 15 21:51:53.178738 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:53.178696    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr" podStartSLOduration=1.1786821490000001 podStartE2EDuration="1.178682149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:51:53.176445751 +0000 UTC m=+2982.533364345" watchObservedRunningTime="2026-07-15 21:51:53.178682149 +0000 UTC m=+2982.535600745"
Jul 15 21:51:53.620734 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:53.620705    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-777b79dc58-2pgbr_11473bef-cb2a-4508-96cd-9163714f1790/console/0.log"
Jul 15 21:51:55.279005 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:55.278952    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-79q2z_c5bef743-7548-4f72-8caa-6798e573e30b/dns/0.log"
Jul 15 21:51:55.300215 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:55.300191    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-79q2z_c5bef743-7548-4f72-8caa-6798e573e30b/kube-rbac-proxy/0.log"
Jul 15 21:51:55.408877 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:55.408847    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-599sr_855e91fe-3c9d-4f36-a073-09eecd10ed4c/dns-node-resolver/0.log"
Jul 15 21:51:56.019493 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:56.019463    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-85bc5987-9hdxw_158e1146-c430-43a7-8217-6bec05abcf73/registry/0.log"
Jul 15 21:51:56.064835 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:56.064803    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lc5h4_0eb4db71-5cbe-49f6-9c8f-99fb32f52a4e/node-ca/0.log"
Jul 15 21:51:57.200373 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:57.200344    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xvvsh_dee496e1-3e51-4ff6-85fb-9058fcf1d770/discovery/0.log"
Jul 15 21:51:57.367427 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:57.367385    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-4lwtr_e3e3752e-0960-4f15-be87-6e1ee6c81472/payload-pre-processing/0.log"
Jul 15 21:51:57.395128 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:57.395092    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-bpf2x_16d252fd-b5c2-4179-9bdd-cde357c9d355/payload-processing/0.log"
Jul 15 21:51:58.161433 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:58.161410    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jv7fg_8da41a2e-7aef-4cc5-87a1-05a9570f8642/serve-healthcheck-canary/0.log"
Jul 15 21:51:58.676849 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:58.676820    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jzqt9_27177627-91b2-4b20-a8ac-7900de8544ca/kube-rbac-proxy/0.log"
Jul 15 21:51:58.697823 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:58.697797    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jzqt9_27177627-91b2-4b20-a8ac-7900de8544ca/exporter/0.log"
Jul 15 21:51:58.717821 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:58.717796    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jzqt9_27177627-91b2-4b20-a8ac-7900de8544ca/extractor/0.log"
Jul 15 21:51:59.174845 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:51:59.174813    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pqvxt/perf-node-gather-daemonset-d7tjr"
Jul 15 21:52:01.944582 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:01.944548    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b45b4f795-9kmmd_8291a711-84ae-42ff-ba75-02528d6525de/manager/0.log"
Jul 15 21:52:01.965190 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:01.965161    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b45b4f795-9kmmd_8291a711-84ae-42ff-ba75-02528d6525de/manager/1.log"
Jul 15 21:52:02.089645 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:02.089610    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-tqlcz_9357ba15-5356-44b6-9451-d10b7c9f0616/server/0.log"
Jul 15 21:52:02.429949 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:02.429917    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c67f598f-qjkkv_e37669e5-eb85-4e79-a7bc-83a947ca45b6/manager/0.log"
Jul 15 21:52:02.474843 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:02.474820    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65c67f598f-qjkkv_e37669e5-eb85-4e79-a7bc-83a947ca45b6/copy-manifests/0.log"
Jul 15 21:52:03.021127 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:03.021095    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xvvsh_dee496e1-3e51-4ff6-85fb-9058fcf1d770/discovery/0.log"
Jul 15 21:52:03.168655 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:03.168626    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-4lwtr_e3e3752e-0960-4f15-be87-6e1ee6c81472/payload-pre-processing/0.log"
Jul 15 21:52:03.191732 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:03.191701    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-bpf2x_16d252fd-b5c2-4179-9bdd-cde357c9d355/payload-processing/0.log"
Jul 15 21:52:03.982139 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:03.982099    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5ccd478559-n6rwj_f9f252d2-aa4b-46fd-b0ae-8bf74c29513c/manager/0.log"
Jul 15 21:52:04.572941 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:04.572910    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xvvsh_dee496e1-3e51-4ff6-85fb-9058fcf1d770/discovery/0.log"
Jul 15 21:52:04.723832 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:04.723808    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-4lwtr_e3e3752e-0960-4f15-be87-6e1ee6c81472/payload-pre-processing/0.log"
Jul 15 21:52:04.753516 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:04.753489    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-bpf2x_16d252fd-b5c2-4179-9bdd-cde357c9d355/payload-processing/0.log"
Jul 15 21:52:09.386161 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:09.386123    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-dx82l_1d90ed07-ed6a-458d-956a-1da3260948d2/kube-storage-version-migrator-operator/1.log"
Jul 15 21:52:09.386870 ip-10-0-130-29 kubenswrapper[2580]: I0715 21:52:09.386854    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-dx82l_1d90ed07-ed6a-458d-956a-1da3260948d2/kube-storage-version-migrator-operator/0.log"