Jul 22 15:04:21.430724 ip-10-0-142-80 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.823273    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827272    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827287    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827292    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827295    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827299    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827302    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827305    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:04:21.832997 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827308    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827311    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827313    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827316    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827324    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827327    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827330    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827333    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827336    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827339    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827341    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827345    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827347    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827350    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827352    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827355    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827357    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827360    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827362    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827365    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:04:21.834314 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827367    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827369    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827373    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827377    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827379    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827382    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827385    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827387    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827390    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827392    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827395    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827397    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827399    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827402    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827405    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827407    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827410    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827412    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827416    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:04:21.835211 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827418    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827421    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827424    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827426    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827430    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827434    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827437    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827440    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827443    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827462    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827465    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827468    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827470    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827473    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827476    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827479    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827482    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827485    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827488    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:04:21.836091 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827490    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827493    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827496    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827498    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827501    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827504    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827507    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827509    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827512    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827514    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827517    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827519    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827522    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827525    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827528    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827531    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827534    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827537    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827539    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827542    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:04:21.836971 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827544    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827547    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827549    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827552    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827554    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827557    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827559    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827562    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827564    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827567    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827570    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827572    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827575    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.827577    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830390    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830407    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830410    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830414    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830417    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830420    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:04:21.837841 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830426    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830429    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830432    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830436    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830440    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830444    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830466    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830469    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830472    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830474    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830477    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830481    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830485    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830488    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830492    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830494    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830498    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830503    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830506    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830508    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:04:21.838712 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830511    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830514    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830516    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830519    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830524    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830527    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830530    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830533    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830536    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830539    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830542    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830544    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830547    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830549    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830557    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830559    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830562    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830571    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830576    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:04:21.839594 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830580    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830583    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830586    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830589    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830592    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830595    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830599    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830603    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830606    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830612    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830615    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830618    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830621    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830624    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830627    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830630    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830632    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830635    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830638    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:04:21.840487 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830641    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830644    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830647    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830652    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830655    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830658    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830661    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830664    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830667    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830669    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830673    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830676    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830678    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830681    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830684    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830690    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830693    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830695    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830698    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830701    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:04:21.841352 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830704    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830707    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830710    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830712    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830715    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830718    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830720    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830725    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830728    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830730    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830733    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830735    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830738    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830740    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.830743    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831369    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831375    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831380    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831383    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:04:21.842239 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831388    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831392    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831395    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831399    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831402    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831409    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831413    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831416    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831419    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831422    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831425    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831427    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831430    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831433    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831435    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831438    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831440    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831463    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831466    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831469    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:04:21.843111 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831471    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831474    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831477    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831480    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831483    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831486    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831489    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831492    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831495    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831498    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831503    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831506    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831509    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831512    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831519    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831522    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831524    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831527    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831530    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831533    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:04:21.843996 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831537    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831542    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831545    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831548    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831551    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831554    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831556    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831559    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831562    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831565    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831568    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831570    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831575    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831578    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831583    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831586    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831589    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831592    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831595    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831598    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:04:21.844882 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831600    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831603    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831606    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831609    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831612    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831619    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831624    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831627    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831629    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831632    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831634    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831637    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831640    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831642    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831646    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831648    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831651    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831654    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831656    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:04:21.846009 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831662    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831665    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831668    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831670    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831673    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831678    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831681    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831683    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831686    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831689    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831691    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831697    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831699    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831702    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831705    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.831708    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831797    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831807    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831813    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831817    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831821    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 15:04:21.846883 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831832    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831837    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831842    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831845    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831851    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831855    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831858    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831862    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831865    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831868    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831871    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831874    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831879    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831883    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831885    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831889    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831894    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831897    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831901    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831904    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831907    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831913    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831916    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831919    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 15:04:21.847749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831922    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831926    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831930    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831940    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831943    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831949    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831952    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831955    2560 flags.go:64] FLAG: --enable-server="true"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831958    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831962    2560 flags.go:64] FLAG: --event-burst="100"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831965    2560 flags.go:64] FLAG: --event-qps="50"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831968    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831971    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831974    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831980    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831983    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831986    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831989    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831993    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831996    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.831999    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832002    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832007    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832010    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832014    2560 flags.go:64] FLAG: --feature-gates=""
Jul 22 15:04:21.848849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832019    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832022    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832025    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832028    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832032    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832037    2560 flags.go:64] FLAG: --help="false"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832040    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-142-80.ec2.internal"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832043    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832046    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832049    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832054    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832058    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832061    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832064    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832069    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832072    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832076    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832079    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832082    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832085    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832088    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832095    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832100    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832103    2560 flags.go:64] FLAG: --lock-file=""
Jul 22 15:04:21.849940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832106    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832109    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832112    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832123    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832125    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832128    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832131    2560 flags.go:64] FLAG: --logging-format="text"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832139    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832143    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832146    2560 flags.go:64] FLAG: --manifest-url=""
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832149    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832153    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832156    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832160    2560 flags.go:64] FLAG: --max-pods="110"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832163    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832169    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832172    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832175    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832179    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832183    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832186    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832193    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832199    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832202    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 15:04:21.851043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832205    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832208    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832213    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832216    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832220    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832223    2560 flags.go:64] FLAG: --port="10250"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832226    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832231    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07c7d2fa4c8466e12"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832235    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832238    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832241    2560 flags.go:64] FLAG: --register-node="true"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832244    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832247    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832250    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832253    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832259    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832265    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832268    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832271    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832274    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832277    2560 flags.go:64] FLAG: --runonce="false"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832280    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832283    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832286    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832291    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 15:04:21.854130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832295    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832300    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832303    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832306    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832309    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832313    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832316    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832321    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832324    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832327    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832330    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832335    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832338    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832341    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832347    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832350    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832353    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832356    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832359    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832362    2560 flags.go:64] FLAG: --v="2"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832366    2560 flags.go:64] FLAG: --version="false"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832370    2560 flags.go:64] FLAG: --vmodule=""
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832376    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.832381    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832602    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:04:21.855226 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832606    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832609    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832612    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832615    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832617    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832620    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832623    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832626    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832630    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832633    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832636    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832641    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832644    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832647    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832650    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832652    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832655    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832658    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832660    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832663    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:04:21.856955 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832665    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832668    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832671    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832674    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832679    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832681    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832685    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832689    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832692    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832694    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832698    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832701    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832703    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832706    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832709    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832711    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832716    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832719    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832726    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832729    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:04:21.857833 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832733    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832736    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832738    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832741    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832744    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832747    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832751    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832755    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832760    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832763    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832766    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832769    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832771    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832774    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832777    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832779    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832782    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832785    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832787    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:04:21.930558 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832790    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832792    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832797    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832801    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832804    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832807    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832809    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832812    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832815    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832818    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832820    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832823    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832826    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832830    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832835    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832837    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832840    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832843    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832845    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832849    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:04:21.931711 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832851    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832854    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832856    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832859    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832862    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832865    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832867    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832872    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832875    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832877    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832884    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832887    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832889    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832892    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832894    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832898    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832901    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832903    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:04:21.937250 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.832907    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.833640    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.866538    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.866555    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.866575    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.866584    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866641    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866646    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866649    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866652    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866655    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866658    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866661    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866665    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866668    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:04:21.938125 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866670    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866673    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866676    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866679    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866681    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866685    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866688    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866690    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866694    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866698    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866701    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866704    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866707    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866709    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866712    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866715    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866718    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866722    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866725    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:04:21.939630 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866727    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866730    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866733    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866736    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866738    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866741    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866743    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866746    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866749    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866752    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866754    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866757    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866759    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866762    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866764    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866767    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866769    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866772    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866774    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866776    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:04:21.940761 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866779    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866782    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866784    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866787    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866789    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866792    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866796    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866798    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866801    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866804    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866806    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866809    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866812    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866814    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866817    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866819    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866822    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866824    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866827    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866830    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:04:21.941989 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866832    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866834    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866837    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866839    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866842    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866846    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866850    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866853    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866856    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866860    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866864    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866867    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866870    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866872    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866875    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866878    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866880    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866883    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866885    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:04:21.946346 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866905    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866909    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866912    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866916    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866919    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866922    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866924    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866927    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866930    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866932    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866935    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.866938    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.866943    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867072    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 15:04:21.947205 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867077    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867080    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867083    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867085    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867088    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867091    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867094    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867096    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867100    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867102    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867105    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867108    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867110    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867113    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867115    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867118    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867120    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867123    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867125    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867128    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 15:04:21.947857 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867131    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867133    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867136    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867139    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867142    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867145    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867147    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867149    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867152    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867156    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867159    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867161    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867164    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867166    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867169    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867173    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867177    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867179    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 15:04:21.949399 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867182    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867185    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867189    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867192    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867195    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867198    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867200    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867203    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867206    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867208    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867211    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867213    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867216    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867218    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867221    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867223    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867226    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867229    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867232    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867235    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 15:04:21.950278 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867237    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867240    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867242    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867245    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867248    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867250    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867253    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867256    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867258    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867261    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867263    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867266    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867268    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867271    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867274    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867277    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867279    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867282    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867284    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867287    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 15:04:21.951139 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867289    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867292    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867294    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867297    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867299    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867303    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867307    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867310    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867312    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867315    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867319    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867322    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867324    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867327    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867330    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867332    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867335    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867337    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867340    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 15:04:21.953110 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:21.867343    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.867347    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.867474    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.869990    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.871634    2560 server.go:1019] "Starting client certificate rotation"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.874070    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.875871    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.875929    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.878474    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.897478    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.899679    2560 log.go:25] "Validated CRI v1 image API"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.900517    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 15:04:21.953985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.902927    2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7bb8dcb4-6916-4ac7-91be-b160730a2211:/dev/nvme0n1p3 c252b537-5d70-4078-9b97-7c286dd4eacd:/dev/nvme0n1p4]
Jul 22 15:04:22.021991 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.902945    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 15:04:21.968336 ip-10-0-142-80 systemd[1]: Started Kubernetes Kubelet.
Jul 22 15:04:22.022484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.908534    2560 manager.go:216] Machine: {Timestamp:2026-07-22 15:04:21.907316607 +0000 UTC m=+0.362030213 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099572 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26d9b95cc7514787f59641b0a78759 SystemUUID:ec26d9b9-5cc7-5147-87f5-9641b0a78759 BootID:f78763c6-1967-4178-9ec3-1dd2d9db243d Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ad:cc:ed:cc:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ad:cc:ed:cc:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:7a:38:6d:2a:18 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 15:04:22.022484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.908627    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 15:04:22.022484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.908743    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 15:04:22.022484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.911522    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 15:04:22.022484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.911559    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-80.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 15:04:22.022484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.912200    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 15:04:22.022484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.912211    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 15:04:22.022484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.912219    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.912984    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.914556    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.917258    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.917367    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.917663    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.918109    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.918130    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.918148    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.918208    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.918230    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.918657    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.918672    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.922874    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.924915    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.924951    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926072    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926086    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926092    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926098    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926111    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926128    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926137    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926143    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926153    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926159    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926167    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926182    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926936    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.926945    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.938335    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-80.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:21.938459    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:21.938476    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.966926    2560 server.go:1268] "Started kubelet"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.967499    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.967753    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g8blm"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.968759    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.968810    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.970311    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 15:04:22.024155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.975349    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.976298    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g8blm"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:21.977119    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:21.976618    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-80.ec2.internal.18c4a4e66f93100b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-80.ec2.internal,UID:ip-10-0-142-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-80.ec2.internal,},FirstTimestamp:2026-07-22 15:04:21.966901259 +0000 UTC m=+0.421614864,LastTimestamp:2026-07-22 15:04:21.966901259 +0000 UTC m=+0.421614864,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-80.ec2.internal,}"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.979543    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.979701    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:21.980382    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.980972    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.980988    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.980973    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981076    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981084    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981109    2560 factory.go:55] Registering systemd factory
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981121    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981338    2560 factory.go:153] Registering CRI-O factory
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981349    2560 factory.go:223] Registration of the crio container factory successfully
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981401    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981426    2560 factory.go:103] Registering Raw factory
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981441    2560 manager.go:1196] Started watching for new ooms in manager
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.981854    2560 manager.go:349] Starting recovery of all containers
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.998328    2560 manager.go:354] Recovery completed
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:21.999425    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.009842    2560 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.012533    2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-80.ec2.internal\" not found" node="ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.012981    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.015766    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.015805    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.015822    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.016334    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.016343    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.016358    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.018091    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.018092    2560 policy_none.go:49] "None policy: Start"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.018183    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.018193    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 15:04:22.027390 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.019225    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:04:22.028431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.019243    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:04:22.028431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.019253    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:04:22.028431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.019310    2560 policy_none.go:47] "Start"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.080464    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.121226    2560 manager.go:337] "Starting Device Plugin manager"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.121411    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.121424    2560 server.go:87] "Starting device plugin registration server"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.121672    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.121682    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.121793    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.121891    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.121902    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.122488    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 15:04:22.127744 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.122539    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.283924 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.222743    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:04:22.283924 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.223925    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:04:22.283924 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.223958    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:04:22.283924 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.223970    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:04:22.283924 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.223998    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.283924 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.233008    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.283924 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.233033    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-80.ec2.internal\": node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.283924 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.266679    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.368390 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.367190    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.520637 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.468139    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.610986 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.568857    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.610986 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.594116    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 15:04:22.610986 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.595537    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 15:04:22.610986 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.595575    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 15:04:22.610986 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.595605    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 15:04:22.610986 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.595651    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 15:04:22.610986 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.598091    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:04:22.671222 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.671198    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.696427 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.696398    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal"]
Jul 22 15:04:22.696500 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.696484    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:04:22.697548 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.697535    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:04:22.697602 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.697564    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:04:22.697602 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.697577    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:04:22.699106 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.699092    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:04:22.699222 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.699205    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.699263 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.699239    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:04:22.700629 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.700611    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:04:22.700695 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.700641    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:04:22.700695 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.700655    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:04:22.700967 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.700951    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:04:22.701023 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.700972    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:04:22.701023 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.700984    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:04:22.702218 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.702199    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.702327 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.702225    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 15:04:22.702843 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.702827    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 15:04:22.702903 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.702853    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 15:04:22.702903 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.702868    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeHasSufficientPID"
Jul 22 15:04:22.722245 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.722223    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-80.ec2.internal\" not found" node="ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.726637 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.726620    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-80.ec2.internal\" not found" node="ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.772086 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.772061    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.786409 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.786385    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.786560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.786414    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.786560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.786437    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ddc4a6bf9a8830c582e31a32617937e6-config\") pod \"kube-apiserver-proxy-ip-10-0-142-80.ec2.internal\" (UID: \"ddc4a6bf9a8830c582e31a32617937e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.869895 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.869820    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:04:22.871315 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.871294    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 15:04:22.871436 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.871418    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 15:04:22.871507 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.871487    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 15:04:22.872489 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:22.872470    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-80.ec2.internal\" not found"
Jul 22 15:04:22.887072 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.887051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.887127 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.887076    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.887127 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.887093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ddc4a6bf9a8830c582e31a32617937e6-config\") pod \"kube-apiserver-proxy-ip-10-0-142-80.ec2.internal\" (UID: \"ddc4a6bf9a8830c582e31a32617937e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.887192 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.887141    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ddc4a6bf9a8830c582e31a32617937e6-config\") pod \"kube-apiserver-proxy-ip-10-0-142-80.ec2.internal\" (UID: \"ddc4a6bf9a8830c582e31a32617937e6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.887192 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.887160    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.887192 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.887174    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35e0fc6a14943475efe90a14e95086ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal\" (UID: \"35e0fc6a14943475efe90a14e95086ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.924881 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.924854    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:04:22.980567 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.980529    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 14:59:22 +0000 UTC" deadline="2027-12-23 05:26:26.019471952 +0000 UTC"
Jul 22 15:04:22.980567 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.980556    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12446h22m3.038919479s"
Jul 22 15:04:22.980567 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.980558    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:22.980769 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.980545    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 15:04:22.993412 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:22.993392    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 15:04:23.000547 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.000526    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 15:04:23.001668 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.001656    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:23.014698 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.014679    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 15:04:23.022941 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.022917    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rmk5k"
Jul 22 15:04:23.026299 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.026287    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:23.030360 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.030202    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal"
Jul 22 15:04:23.031252 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.031236    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rmk5k"
Jul 22 15:04:23.904987 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.904955    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:04:23.919196 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.919165    2560 apiserver.go:52] "Watching apiserver"
Jul 22 15:04:23.931670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.931639    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 15:04:23.932663 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.932638    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-8ckgj","openshift-network-operator/iptables-alerter-dt7cb","openshift-ovn-kubernetes/ovnkube-node-wf658","kube-system/konnectivity-agent-4nbnn","kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal","openshift-cluster-node-tuning-operator/tuned-bjq6l","openshift-dns/node-resolver-z4765","openshift-image-registry/node-ca-hkpl4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal","openshift-multus/multus-45g86","openshift-multus/network-metrics-daemon-w9tkp","kube-system/global-pull-secret-syncer-c8jkn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6","openshift-multus/multus-additional-cni-plugins-d8g44"]
Jul 22 15:04:23.934438 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.934419    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:23.936225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.936206    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:23.936325 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:23.936272    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:23.937196 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.937179    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 15:04:23.937279 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.937262    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w5sfb\""
Jul 22 15:04:23.937320 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.937273    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 15:04:23.937551 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.937537    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.938783 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.938764    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:23.940134 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.940118    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:23.940769 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.940753    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 15:04:23.940941 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.940928    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 15:04:23.941053 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.940986    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 15:04:23.941297 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.941275    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ltqnk\""
Jul 22 15:04:23.941410 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.941392    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.942291 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.942275    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 15:04:23.942383 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.942300    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 15:04:23.942514 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.942495    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 15:04:23.942635 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.942565    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 15:04:23.942733 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.942718    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 15:04:23.942811 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.942767    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7mbh2\""
Jul 22 15:04:23.942811 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.942804    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zlw6r\""
Jul 22 15:04:23.942918 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.942867    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:23.943783 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.943764    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 15:04:23.943901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.943825    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 15:04:23.943901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.943827    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 15:04:23.943901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.943894    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 15:04:23.944042 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.944014    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 15:04:23.944042 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.944038    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7xz7d\""
Jul 22 15:04:23.944145 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.944064    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 15:04:23.944145 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.944084    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 15:04:23.945142 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.945127    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 15:04:23.945522 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.945509    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:23.947519 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.947497    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:23.948882 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.948865    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.950144 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.950129    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:23.950207 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:23.950188    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:23.951667 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.951641    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 15:04:23.951937 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.951917    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 15:04:23.952003 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.951950    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 15:04:23.952239 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.952220    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4ptp6\""
Jul 22 15:04:23.952334 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.952266    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 15:04:23.953320 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.952673    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:23.953320 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:23.952753    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:23.953320 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.952832    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sj7ck\""
Jul 22 15:04:23.958653 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.958624    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 15:04:23.958754 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.958722    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 15:04:23.958840 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.958824    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 15:04:23.959715 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.959696    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 15:04:23.959795 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.959712    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qgcqv\""
Jul 22 15:04:23.959795 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.959712    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5mk7g\""
Jul 22 15:04:23.959890 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.959716    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 15:04:23.982130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.982113    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 15:04:23.993804 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.993783    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-slash\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.993912 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.993812    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-run-openvswitch\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.993912 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.993829    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-var-lib-kubelet\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.993912 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.993852    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa399725-34cb-4e9a-b711-97b1178f7915-etc-tuned\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.993912 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.993876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/871c5594-a183-4116-8d8a-20ee028153f5-cni-binary-copy\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.993912 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.993900    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-kubelet\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994102 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.993969    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-systemd-units\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994102 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994004    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-var-lib-openvswitch\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994102 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994025    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-run-ovn\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994102 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994051    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-node-log\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994102 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994074    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994102 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994092    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-run\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.994354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b0d8426-5e95-49b9-b057-b5e5cfc59182-dbus\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:23.994354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994131    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78pl8\" (UniqueName: \"kubernetes.io/projected/dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f-kube-api-access-78pl8\") pod \"node-resolver-z4765\" (UID: \"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f\") " pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:23.994354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994149    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9d0d4357-a4b6-43fb-8587-4b50695f40e5-env-overrides\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994182    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-sysconfig\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.994354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994207    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-sys\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.994354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994235    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x5tw\" (UniqueName: \"kubernetes.io/projected/aa399725-34cb-4e9a-b711-97b1178f7915-kube-api-access-7x5tw\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.994354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994264    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-multus-socket-dir-parent\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.994354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994321    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e608c87c-656a-470f-9bb6-d2125d19fd29-host-slash\") pod \"iptables-alerter-dt7cb\" (UID: \"e608c87c-656a-470f-9bb6-d2125d19fd29\") " pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994362    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-multus-cni-dir\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994389    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f-tmp-dir\") pod \"node-resolver-z4765\" (UID: \"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f\") " pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994422    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckhjd\" (UniqueName: \"kubernetes.io/projected/871c5594-a183-4116-8d8a-20ee028153f5-kube-api-access-ckhjd\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994449    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-sys-fs\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994505    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-cni-netd\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994531    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-sysctl-d\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994557    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994582    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-etc-openvswitch\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994619    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa399725-34cb-4e9a-b711-97b1178f7915-tmp\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994642    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d1e6293-7310-4977-81ce-f804acdcbcd0-host\") pod \"node-ca-hkpl4\" (UID: \"3d1e6293-7310-4977-81ce-f804acdcbcd0\") " pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994666    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-run-netns\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-run-multus-certs\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-system-cni-dir\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:23.994833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994751    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-socket-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994788    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-cni-bin\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994815    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72t5z\" (UniqueName: \"kubernetes.io/projected/9d0d4357-a4b6-43fb-8587-4b50695f40e5-kube-api-access-72t5z\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994839    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-systemd\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994863    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994889    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12e87e9c-0d5e-4be3-99be-110478935005-konnectivity-ca\") pod \"konnectivity-agent-4nbnn\" (UID: \"12e87e9c-0d5e-4be3-99be-110478935005\") " pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994938    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994966    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1e56680-b7ed-4958-848e-7c7e0e75c826-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.994994    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9d0d4357-a4b6-43fb-8587-4b50695f40e5-ovnkube-script-lib\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995031    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-cnibin\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-os-release\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995099    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-device-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995122    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-run-netns\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.995502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-lib-modules\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995171    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-cnibin\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995197    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-hostroot\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995270    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/871c5594-a183-4116-8d8a-20ee028153f5-multus-daemon-config\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995308    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b0d8426-5e95-49b9-b057-b5e5cfc59182-kubelet-config\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995336    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-sysctl-conf\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995363    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-host\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995396    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-run-k8s-cni-cncf-io\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbcvl\" (UniqueName: \"kubernetes.io/projected/ba27779c-af36-4803-88e3-0f0f25a01210-kube-api-access-xbcvl\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995456    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1e56680-b7ed-4958-848e-7c7e0e75c826-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995486    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-modprobe-d\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995509    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-kubernetes\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995556    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-os-release\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85vlx\" (UniqueName: \"kubernetes.io/projected/e608c87c-656a-470f-9bb6-d2125d19fd29-kube-api-access-85vlx\") pod \"iptables-alerter-dt7cb\" (UID: \"e608c87c-656a-470f-9bb6-d2125d19fd29\") " pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:23.995990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995644    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-run-systemd\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995670    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3d1e6293-7310-4977-81ce-f804acdcbcd0-serviceca\") pod \"node-ca-hkpl4\" (UID: \"3d1e6293-7310-4977-81ce-f804acdcbcd0\") " pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995686    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1e56680-b7ed-4958-848e-7c7e0e75c826-cni-binary-copy\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995704    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbnsl\" (UniqueName: \"kubernetes.io/projected/538d0762-b726-4179-83e0-e5028e629c65-kube-api-access-qbnsl\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995735    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9d0d4357-a4b6-43fb-8587-4b50695f40e5-ovnkube-config\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995749    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-etc-kubernetes\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-etc-selinux\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f-hosts-file\") pod \"node-resolver-z4765\" (UID: \"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f\") " pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995794    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e608c87c-656a-470f-9bb6-d2125d19fd29-iptables-alerter-script\") pod \"iptables-alerter-dt7cb\" (UID: \"e608c87c-656a-470f-9bb6-d2125d19fd29\") " pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995811    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-var-lib-cni-bin\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995826    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-registration-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995840    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-log-socket\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.996398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995855    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9d0d4357-a4b6-43fb-8587-4b50695f40e5-ovn-node-metrics-cert\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:23.996812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:23.996812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995903    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v98d\" (UniqueName: \"kubernetes.io/projected/3d1e6293-7310-4977-81ce-f804acdcbcd0-kube-api-access-9v98d\") pod \"node-ca-hkpl4\" (UID: \"3d1e6293-7310-4977-81ce-f804acdcbcd0\") " pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:23.996812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995918    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-system-cni-dir\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.996812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995934    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12e87e9c-0d5e-4be3-99be-110478935005-agent-certs\") pod \"konnectivity-agent-4nbnn\" (UID: \"12e87e9c-0d5e-4be3-99be-110478935005\") " pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:23.996812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995950    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf26w\" (UniqueName: \"kubernetes.io/projected/b1e56680-b7ed-4958-848e-7c7e0e75c826-kube-api-access-zf26w\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:23.996812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995969    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-var-lib-cni-multus\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.996812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.995989    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-var-lib-kubelet\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:23.996812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:23.996004    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-multus-conf-dir\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.032967 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.032922    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 14:59:23 +0000 UTC" deadline="2028-04-02 23:53:17.350668912 +0000 UTC"
Jul 22 15:04:24.032967 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.032965    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14888h48m53.317708021s"
Jul 22 15:04:24.096896 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.096871    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12e87e9c-0d5e-4be3-99be-110478935005-agent-certs\") pod \"konnectivity-agent-4nbnn\" (UID: \"12e87e9c-0d5e-4be3-99be-110478935005\") " pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:24.097019 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.096909    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf26w\" (UniqueName: \"kubernetes.io/projected/b1e56680-b7ed-4958-848e-7c7e0e75c826-kube-api-access-zf26w\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.097019 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.096934    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-var-lib-cni-multus\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.097019 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.096981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-var-lib-kubelet\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.097195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097013    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-multus-conf-dir\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.097195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097048    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-slash\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-run-openvswitch\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097107    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-var-lib-kubelet\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.097195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097117    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-var-lib-kubelet\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.097195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097118    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-multus-conf-dir\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.097195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097089    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-var-lib-cni-multus\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.097195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-slash\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097144    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa399725-34cb-4e9a-b711-97b1178f7915-etc-tuned\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.097544 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097196    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-run-openvswitch\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097544 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097202    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/871c5594-a183-4116-8d8a-20ee028153f5-cni-binary-copy\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.097544 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097215    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-var-lib-kubelet\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.097544 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097327    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-kubelet\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097544 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097336    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 15:04:24.097544 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097399    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-kubelet\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097544 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-systemd-units\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097544 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097480    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-var-lib-openvswitch\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097544 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097526    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-systemd-units\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097540    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-run-ovn\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097569    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-var-lib-openvswitch\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097582    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-node-log\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097624    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-node-log\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097619    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-run-ovn\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097648    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097690    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-run\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097700    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097716    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b0d8426-5e95-49b9-b057-b5e5cfc59182-dbus\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097737    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78pl8\" (UniqueName: \"kubernetes.io/projected/dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f-kube-api-access-78pl8\") pod \"node-resolver-z4765\" (UID: \"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f\") " pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097753    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097770    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9d0d4357-a4b6-43fb-8587-4b50695f40e5-env-overrides\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097794    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-sysconfig\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.097943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097819    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-sys\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097829    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b0d8426-5e95-49b9-b057-b5e5cfc59182-dbus\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097864    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-run\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097844    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x5tw\" (UniqueName: \"kubernetes.io/projected/aa399725-34cb-4e9a-b711-97b1178f7915-kube-api-access-7x5tw\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097906    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/871c5594-a183-4116-8d8a-20ee028153f5-cni-binary-copy\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097924    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-sysconfig\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097930    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-multus-socket-dir-parent\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-multus-socket-dir-parent\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.097981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e608c87c-656a-470f-9bb6-d2125d19fd29-host-slash\") pod \"iptables-alerter-dt7cb\" (UID: \"e608c87c-656a-470f-9bb6-d2125d19fd29\") " pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098058    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-sys\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098103    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-multus-cni-dir\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e608c87c-656a-470f-9bb6-d2125d19fd29-host-slash\") pod \"iptables-alerter-dt7cb\" (UID: \"e608c87c-656a-470f-9bb6-d2125d19fd29\") " pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098206    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f-tmp-dir\") pod \"node-resolver-z4765\" (UID: \"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f\") " pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098275    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-multus-cni-dir\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.098560 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098277    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckhjd\" (UniqueName: \"kubernetes.io/projected/871c5594-a183-4116-8d8a-20ee028153f5-kube-api-access-ckhjd\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-sys-fs\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098347    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-cni-netd\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-sysctl-d\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098408    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-sys-fs\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098416    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098443    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-etc-openvswitch\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-cni-netd\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098483    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa399725-34cb-4e9a-b711-97b1178f7915-tmp\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098523    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d1e6293-7310-4977-81ce-f804acdcbcd0-host\") pod \"node-ca-hkpl4\" (UID: \"3d1e6293-7310-4977-81ce-f804acdcbcd0\") " pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098564    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f-tmp-dir\") pod \"node-resolver-z4765\" (UID: \"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f\") " pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098574    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-run-netns\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-run-multus-certs\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098675    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-system-cni-dir\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-socket-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.099349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098715    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-sysctl-d\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098774    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-cni-bin\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098792    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d1e6293-7310-4977-81ce-f804acdcbcd0-host\") pod \"node-ca-hkpl4\" (UID: \"3d1e6293-7310-4977-81ce-f804acdcbcd0\") " pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098800    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72t5z\" (UniqueName: \"kubernetes.io/projected/9d0d4357-a4b6-43fb-8587-4b50695f40e5-kube-api-access-72t5z\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-etc-openvswitch\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098879    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-systemd\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098886    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9d0d4357-a4b6-43fb-8587-4b50695f40e5-env-overrides\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098905    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.098955    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-system-cni-dir\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099003    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-run-multus-certs\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099005    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-run-netns\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099041    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12e87e9c-0d5e-4be3-99be-110478935005-konnectivity-ca\") pod \"konnectivity-agent-4nbnn\" (UID: \"12e87e9c-0d5e-4be3-99be-110478935005\") " pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.099050    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099072    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099094    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-systemd\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.100075 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099105    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1e56680-b7ed-4958-848e-7c7e0e75c826-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.099138    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret podName:2b0d8426-5e95-49b9-b057-b5e5cfc59182 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:24.59911667 +0000 UTC m=+3.053830280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret") pod "global-pull-secret-syncer-c8jkn" (UID: "2b0d8426-5e95-49b9-b057-b5e5cfc59182") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099164    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-socket-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099170    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9d0d4357-a4b6-43fb-8587-4b50695f40e5-ovnkube-script-lib\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-cnibin\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099235    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-cni-bin\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099267    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-os-release\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099307    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-device-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099344    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-run-netns\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099391    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-lib-modules\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099400    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-cnibin\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099455    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-hostroot\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099475    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-cnibin\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.100897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/871c5594-a183-4116-8d8a-20ee028153f5-multus-daemon-config\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099520    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b0d8426-5e95-49b9-b057-b5e5cfc59182-kubelet-config\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099571    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-sysctl-conf\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099610    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-host\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-run-k8s-cni-cncf-io\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099661    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbcvl\" (UniqueName: \"kubernetes.io/projected/ba27779c-af36-4803-88e3-0f0f25a01210-kube-api-access-xbcvl\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1e56680-b7ed-4958-848e-7c7e0e75c826-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099718    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-modprobe-d\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099737    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12e87e9c-0d5e-4be3-99be-110478935005-konnectivity-ca\") pod \"konnectivity-agent-4nbnn\" (UID: \"12e87e9c-0d5e-4be3-99be-110478935005\") " pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099723    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-hostroot\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-kubernetes\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099795    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1e56680-b7ed-4958-848e-7c7e0e75c826-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099811    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1e56680-b7ed-4958-848e-7c7e0e75c826-os-release\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099790    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-os-release\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.101478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099850    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-host-run-netns\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85vlx\" (UniqueName: \"kubernetes.io/projected/e608c87c-656a-470f-9bb6-d2125d19fd29-kube-api-access-85vlx\") pod \"iptables-alerter-dt7cb\" (UID: \"e608c87c-656a-470f-9bb6-d2125d19fd29\") " pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099864    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b0d8426-5e95-49b9-b057-b5e5cfc59182-kubelet-config\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099899    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-device-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099896    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-run-systemd\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099944    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-lib-modules\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099953    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3d1e6293-7310-4977-81ce-f804acdcbcd0-serviceca\") pod \"node-ca-hkpl4\" (UID: \"3d1e6293-7310-4977-81ce-f804acdcbcd0\") " pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.099959    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-cnibin\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100015    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-host\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100188    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9d0d4357-a4b6-43fb-8587-4b50695f40e5-ovnkube-script-lib\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100188    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-run-k8s-cni-cncf-io\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-run-systemd\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1e56680-b7ed-4958-848e-7c7e0e75c826-cni-binary-copy\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100301    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-modprobe-d\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.102208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100308    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100336    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnsl\" (UniqueName: \"kubernetes.io/projected/538d0762-b726-4179-83e0-e5028e629c65-kube-api-access-qbnsl\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100371    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-sysctl-conf\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100376    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9d0d4357-a4b6-43fb-8587-4b50695f40e5-ovnkube-config\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100414    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/871c5594-a183-4116-8d8a-20ee028153f5-multus-daemon-config\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100416    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-etc-kubernetes\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100449    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-etc-kubernetes\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100475    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-etc-selinux\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100481    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3d1e6293-7310-4977-81ce-f804acdcbcd0-serviceca\") pod \"node-ca-hkpl4\" (UID: \"3d1e6293-7310-4977-81ce-f804acdcbcd0\") " pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100501    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f-hosts-file\") pod \"node-resolver-z4765\" (UID: \"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f\") " pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100555    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e608c87c-656a-470f-9bb6-d2125d19fd29-iptables-alerter-script\") pod \"iptables-alerter-dt7cb\" (UID: \"e608c87c-656a-470f-9bb6-d2125d19fd29\") " pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100591    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-var-lib-cni-bin\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100620    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-registration-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100660    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-log-socket\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.102950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9d0d4357-a4b6-43fb-8587-4b50695f40e5-ovn-node-metrics-cert\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100718    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100746    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v98d\" (UniqueName: \"kubernetes.io/projected/3d1e6293-7310-4977-81ce-f804acdcbcd0-kube-api-access-9v98d\") pod \"node-ca-hkpl4\" (UID: \"3d1e6293-7310-4977-81ce-f804acdcbcd0\") " pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100777    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-system-cni-dir\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100818    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1e56680-b7ed-4958-848e-7c7e0e75c826-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100826    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-host-var-lib-cni-bin\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100882    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-system-cni-dir\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.100941    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f-hosts-file\") pod \"node-resolver-z4765\" (UID: \"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f\") " pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.101455    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9d0d4357-a4b6-43fb-8587-4b50695f40e5-ovnkube-config\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.101480    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/871c5594-a183-4116-8d8a-20ee028153f5-os-release\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.101505    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1e56680-b7ed-4958-848e-7c7e0e75c826-cni-binary-copy\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.101562    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-etc-selinux\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.101582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/538d0762-b726-4179-83e0-e5028e629c65-registration-dir\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.101607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa399725-34cb-4e9a-b711-97b1178f7915-etc-kubernetes\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.103516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.101612    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9d0d4357-a4b6-43fb-8587-4b50695f40e5-log-socket\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.103985 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.101681    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:24.103985 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.101737    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs podName:ba27779c-af36-4803-88e3-0f0f25a01210 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:24.601719187 +0000 UTC m=+3.056432786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs") pod "network-metrics-daemon-w9tkp" (UID: "ba27779c-af36-4803-88e3-0f0f25a01210") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:24.103985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.101944    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa399725-34cb-4e9a-b711-97b1178f7915-etc-tuned\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.103985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.102081    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e608c87c-656a-470f-9bb6-d2125d19fd29-iptables-alerter-script\") pod \"iptables-alerter-dt7cb\" (UID: \"e608c87c-656a-470f-9bb6-d2125d19fd29\") " pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:24.103985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.102255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12e87e9c-0d5e-4be3-99be-110478935005-agent-certs\") pod \"konnectivity-agent-4nbnn\" (UID: \"12e87e9c-0d5e-4be3-99be-110478935005\") " pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:24.103985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.102679    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa399725-34cb-4e9a-b711-97b1178f7915-tmp\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.103985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.103439    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9d0d4357-a4b6-43fb-8587-4b50695f40e5-ovn-node-metrics-cert\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.113268 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.113240    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:04:24.113268 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.113271    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:04:24.113410 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.113287    2560 projected.go:196] Error preparing data for projected volume kube-api-access-dnh2n for pod openshift-network-diagnostics/network-check-target-8ckgj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:24.113410 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.113354    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n podName:35ef3246-ba7b-44fd-8a27-d59603445a9a nodeName:}" failed. No retries permitted until 2026-07-22 15:04:24.613332559 +0000 UTC m=+3.068046178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dnh2n" (UniqueName: "kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n") pod "network-check-target-8ckgj" (UID: "35ef3246-ba7b-44fd-8a27-d59603445a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:24.114733 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.114580    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf26w\" (UniqueName: \"kubernetes.io/projected/b1e56680-b7ed-4958-848e-7c7e0e75c826-kube-api-access-zf26w\") pod \"multus-additional-cni-plugins-d8g44\" (UID: \"b1e56680-b7ed-4958-848e-7c7e0e75c826\") " pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.114733 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.114607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckhjd\" (UniqueName: \"kubernetes.io/projected/871c5594-a183-4116-8d8a-20ee028153f5-kube-api-access-ckhjd\") pod \"multus-45g86\" (UID: \"871c5594-a183-4116-8d8a-20ee028153f5\") " pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.114905 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.114730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x5tw\" (UniqueName: \"kubernetes.io/projected/aa399725-34cb-4e9a-b711-97b1178f7915-kube-api-access-7x5tw\") pod \"tuned-bjq6l\" (UID: \"aa399725-34cb-4e9a-b711-97b1178f7915\") " pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.115020 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.115001    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78pl8\" (UniqueName: \"kubernetes.io/projected/dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f-kube-api-access-78pl8\") pod \"node-resolver-z4765\" (UID: \"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f\") " pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:24.115610 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.115575    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbcvl\" (UniqueName: \"kubernetes.io/projected/ba27779c-af36-4803-88e3-0f0f25a01210-kube-api-access-xbcvl\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:24.115824 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.115802    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v98d\" (UniqueName: \"kubernetes.io/projected/3d1e6293-7310-4977-81ce-f804acdcbcd0-kube-api-access-9v98d\") pod \"node-ca-hkpl4\" (UID: \"3d1e6293-7310-4977-81ce-f804acdcbcd0\") " pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:24.116716 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.116695    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbnsl\" (UniqueName: \"kubernetes.io/projected/538d0762-b726-4179-83e0-e5028e629c65-kube-api-access-qbnsl\") pod \"aws-ebs-csi-driver-node-sbgt6\" (UID: \"538d0762-b726-4179-83e0-e5028e629c65\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.116803 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.116728    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85vlx\" (UniqueName: \"kubernetes.io/projected/e608c87c-656a-470f-9bb6-d2125d19fd29-kube-api-access-85vlx\") pod \"iptables-alerter-dt7cb\" (UID: \"e608c87c-656a-470f-9bb6-d2125d19fd29\") " pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:24.116928 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.116905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72t5z\" (UniqueName: \"kubernetes.io/projected/9d0d4357-a4b6-43fb-8587-4b50695f40e5-kube-api-access-72t5z\") pod \"ovnkube-node-wf658\" (UID: \"9d0d4357-a4b6-43fb-8587-4b50695f40e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.118818 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.118796    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 15:04:24.263810 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.263718    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z4765"
Jul 22 15:04:24.279206 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.279179    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:24.293383 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.293358    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:24.297154 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.297131    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d8g44"
Jul 22 15:04:24.317286 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.317257    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bjq6l"
Jul 22 15:04:24.334190 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.334161    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hkpl4"
Jul 22 15:04:24.338527 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.338445    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6"
Jul 22 15:04:24.368161 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.368124    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dt7cb"
Jul 22 15:04:24.374324 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.374301    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-45g86"
Jul 22 15:04:24.604340 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.604268    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:24.604340 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.604320    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:24.604584 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.604423    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:24.604584 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.604441    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:24.604584 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.604492    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret podName:2b0d8426-5e95-49b9-b057-b5e5cfc59182 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:25.604476861 +0000 UTC m=+4.059190469 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret") pod "global-pull-secret-syncer-c8jkn" (UID: "2b0d8426-5e95-49b9-b057-b5e5cfc59182") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:24.604584 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.604516    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs podName:ba27779c-af36-4803-88e3-0f0f25a01210 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:25.604498942 +0000 UTC m=+4.059212554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs") pod "network-metrics-daemon-w9tkp" (UID: "ba27779c-af36-4803-88e3-0f0f25a01210") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:24.705141 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.705109    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:24.705345 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.705325    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:04:24.705414 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.705354    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:04:24.705414 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.705368    2560 projected.go:196] Error preparing data for projected volume kube-api-access-dnh2n for pod openshift-network-diagnostics/network-check-target-8ckgj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:24.705539 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:24.705445    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n podName:35ef3246-ba7b-44fd-8a27-d59603445a9a nodeName:}" failed. No retries permitted until 2026-07-22 15:04:25.70542409 +0000 UTC m=+4.160137716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnh2n" (UniqueName: "kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n") pod "network-check-target-8ckgj" (UID: "35ef3246-ba7b-44fd-8a27-d59603445a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:24.827234 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:24.827197    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode608c87c_656a_470f_9bb6_d2125d19fd29.slice/crio-a7125e5511b6af1b9594bb2c1cc8b277679194a91a5690432261ca5488764f4b WatchSource:0}: Error finding container a7125e5511b6af1b9594bb2c1cc8b277679194a91a5690432261ca5488764f4b: Status 404 returned error can't find the container with id a7125e5511b6af1b9594bb2c1cc8b277679194a91a5690432261ca5488764f4b
Jul 22 15:04:24.827851 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:24.827823    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d0d4357_a4b6_43fb_8587_4b50695f40e5.slice/crio-9498e4efe010f38e2ad2fc4852f4a11f5b026394a014a95d73b64654946c68fe WatchSource:0}: Error finding container 9498e4efe010f38e2ad2fc4852f4a11f5b026394a014a95d73b64654946c68fe: Status 404 returned error can't find the container with id 9498e4efe010f38e2ad2fc4852f4a11f5b026394a014a95d73b64654946c68fe
Jul 22 15:04:24.829084 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:24.829056    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538d0762_b726_4179_83e0_e5028e629c65.slice/crio-e289555e584e70dc27c71cd62e604bf84561383fa11afee45c50c3a711a39179 WatchSource:0}: Error finding container e289555e584e70dc27c71cd62e604bf84561383fa11afee45c50c3a711a39179: Status 404 returned error can't find the container with id e289555e584e70dc27c71cd62e604bf84561383fa11afee45c50c3a711a39179
Jul 22 15:04:24.829727 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:24.829691    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd87a8d8_cc8e_4b2c_82c7_a9b81a5c016f.slice/crio-014474cb4263989ee3a7175d28baf69bcfc4da2fc0fab65477e6ece7a7550d9b WatchSource:0}: Error finding container 014474cb4263989ee3a7175d28baf69bcfc4da2fc0fab65477e6ece7a7550d9b: Status 404 returned error can't find the container with id 014474cb4263989ee3a7175d28baf69bcfc4da2fc0fab65477e6ece7a7550d9b
Jul 22 15:04:24.838129 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:24.838111    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:04:24.844158 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:24.844140    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871c5594_a183_4116_8d8a_20ee028153f5.slice/crio-96c1a5b683d41476569bd507e11ac536042e0a552051445e3c84e00d5121e142 WatchSource:0}: Error finding container 96c1a5b683d41476569bd507e11ac536042e0a552051445e3c84e00d5121e142: Status 404 returned error can't find the container with id 96c1a5b683d41476569bd507e11ac536042e0a552051445e3c84e00d5121e142
Jul 22 15:04:24.862087 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:24.862063    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d1e6293_7310_4977_81ce_f804acdcbcd0.slice/crio-4f4dbdffae37f29ca48c940f9b10f9c9c01b26d7731c0d0e3ac18643d0b4e9a4 WatchSource:0}: Error finding container 4f4dbdffae37f29ca48c940f9b10f9c9c01b26d7731c0d0e3ac18643d0b4e9a4: Status 404 returned error can't find the container with id 4f4dbdffae37f29ca48c940f9b10f9c9c01b26d7731c0d0e3ac18643d0b4e9a4
Jul 22 15:04:24.890251 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:24.890230    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e56680_b7ed_4958_848e_7c7e0e75c826.slice/crio-c13bb69859dccb886bf0a91146b6145d3696edabfccd55346260ea50cc14816d WatchSource:0}: Error finding container c13bb69859dccb886bf0a91146b6145d3696edabfccd55346260ea50cc14816d: Status 404 returned error can't find the container with id c13bb69859dccb886bf0a91146b6145d3696edabfccd55346260ea50cc14816d
Jul 22 15:04:25.033558 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.033499    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 14:59:23 +0000 UTC" deadline="2028-03-28 14:28:29.259853759 +0000 UTC"
Jul 22 15:04:25.033558 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.033550    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14759h24m4.226306686s"
Jul 22 15:04:25.061003 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:25.060976    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e87e9c_0d5e_4be3_99be_110478935005.slice/crio-458436b56103c093f6b5578fcb0fa82c0048462a20f1a2069a9eef9417d80947 WatchSource:0}: Error finding container 458436b56103c093f6b5578fcb0fa82c0048462a20f1a2069a9eef9417d80947: Status 404 returned error can't find the container with id 458436b56103c093f6b5578fcb0fa82c0048462a20f1a2069a9eef9417d80947
Jul 22 15:04:25.113881 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:25.113855    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa399725_34cb_4e9a_b711_97b1178f7915.slice/crio-62adc90b99f1919428d0db91748061bb97f99523fe72c16a28997185b18d31de WatchSource:0}: Error finding container 62adc90b99f1919428d0db91748061bb97f99523fe72c16a28997185b18d31de: Status 404 returned error can't find the container with id 62adc90b99f1919428d0db91748061bb97f99523fe72c16a28997185b18d31de
Jul 22 15:04:25.534321 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:25.534281    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e0fc6a14943475efe90a14e95086ef.slice/crio-640ab544a6040910e0afb2a17ee3402e8dd869826ade7c0ecba705a963e7139b WatchSource:0}: Error finding container 640ab544a6040910e0afb2a17ee3402e8dd869826ade7c0ecba705a963e7139b: Status 404 returned error can't find the container with id 640ab544a6040910e0afb2a17ee3402e8dd869826ade7c0ecba705a963e7139b
Jul 22 15:04:25.597422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.596612    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:25.597422 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.596738    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:25.597422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.597152    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:25.597422 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.597255    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:25.597422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.597310    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:25.597422 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.597377    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:25.609726 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.609673    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" event={"ID":"35e0fc6a14943475efe90a14e95086ef","Type":"ContainerStarted","Data":"640ab544a6040910e0afb2a17ee3402e8dd869826ade7c0ecba705a963e7139b"}
Jul 22 15:04:25.611482 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.611441    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:25.611592 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.611532    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:25.611659 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.611641    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:25.611728 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.611701    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret podName:2b0d8426-5e95-49b9-b057-b5e5cfc59182 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:27.611680381 +0000 UTC m=+6.066393996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret") pod "global-pull-secret-syncer-c8jkn" (UID: "2b0d8426-5e95-49b9-b057-b5e5cfc59182") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:25.611799 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.611766    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:25.611853 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.611803    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs podName:ba27779c-af36-4803-88e3-0f0f25a01210 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:27.611791039 +0000 UTC m=+6.066504645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs") pod "network-metrics-daemon-w9tkp" (UID: "ba27779c-af36-4803-88e3-0f0f25a01210") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:25.615141 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.615113    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" event={"ID":"ddc4a6bf9a8830c582e31a32617937e6","Type":"ContainerStarted","Data":"de72cd7421509aad8a8713c314908bae86dcdf28bc26ce3a04dced0c64a4c608"}
Jul 22 15:04:25.618728 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.618702    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4nbnn" event={"ID":"12e87e9c-0d5e-4be3-99be-110478935005","Type":"ContainerStarted","Data":"458436b56103c093f6b5578fcb0fa82c0048462a20f1a2069a9eef9417d80947"}
Jul 22 15:04:25.621878 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.621845    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8g44" event={"ID":"b1e56680-b7ed-4958-848e-7c7e0e75c826","Type":"ContainerStarted","Data":"c13bb69859dccb886bf0a91146b6145d3696edabfccd55346260ea50cc14816d"}
Jul 22 15:04:25.633228 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.633202    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-45g86" event={"ID":"871c5594-a183-4116-8d8a-20ee028153f5","Type":"ContainerStarted","Data":"96c1a5b683d41476569bd507e11ac536042e0a552051445e3c84e00d5121e142"}
Jul 22 15:04:25.637222 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.637143    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dt7cb" event={"ID":"e608c87c-656a-470f-9bb6-d2125d19fd29","Type":"ContainerStarted","Data":"a7125e5511b6af1b9594bb2c1cc8b277679194a91a5690432261ca5488764f4b"}
Jul 22 15:04:25.640508 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.640485    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerStarted","Data":"9498e4efe010f38e2ad2fc4852f4a11f5b026394a014a95d73b64654946c68fe"}
Jul 22 15:04:25.643153 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.643114    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6" event={"ID":"538d0762-b726-4179-83e0-e5028e629c65","Type":"ContainerStarted","Data":"e289555e584e70dc27c71cd62e604bf84561383fa11afee45c50c3a711a39179"}
Jul 22 15:04:25.645384 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.645361    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bjq6l" event={"ID":"aa399725-34cb-4e9a-b711-97b1178f7915","Type":"ContainerStarted","Data":"62adc90b99f1919428d0db91748061bb97f99523fe72c16a28997185b18d31de"}
Jul 22 15:04:25.649852 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.649828    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hkpl4" event={"ID":"3d1e6293-7310-4977-81ce-f804acdcbcd0","Type":"ContainerStarted","Data":"4f4dbdffae37f29ca48c940f9b10f9c9c01b26d7731c0d0e3ac18643d0b4e9a4"}
Jul 22 15:04:25.659790 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.659766    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z4765" event={"ID":"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f","Type":"ContainerStarted","Data":"014474cb4263989ee3a7175d28baf69bcfc4da2fc0fab65477e6ece7a7550d9b"}
Jul 22 15:04:25.712378 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:25.712292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:25.712555 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.712531    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:04:25.712628 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.712559    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:04:25.712628 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.712573    2560 projected.go:196] Error preparing data for projected volume kube-api-access-dnh2n for pod openshift-network-diagnostics/network-check-target-8ckgj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:25.712724 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:25.712641    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n podName:35ef3246-ba7b-44fd-8a27-d59603445a9a nodeName:}" failed. No retries permitted until 2026-07-22 15:04:27.712620158 +0000 UTC m=+6.167333771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnh2n" (UniqueName: "kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n") pod "network-check-target-8ckgj" (UID: "35ef3246-ba7b-44fd-8a27-d59603445a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:27.595805 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:27.595773    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:27.596391 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.595910    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:27.596391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:27.596382    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:27.596594 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.596498    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:27.596594 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:27.596544    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:27.596694 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.596617    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:27.633607 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:27.632904    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:27.633607 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:27.632969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:27.633607 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.633099    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:27.633607 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.633161    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs podName:ba27779c-af36-4803-88e3-0f0f25a01210 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:31.633140246 +0000 UTC m=+10.087853851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs") pod "network-metrics-daemon-w9tkp" (UID: "ba27779c-af36-4803-88e3-0f0f25a01210") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:27.633607 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.633522    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:27.633607 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.633572    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret podName:2b0d8426-5e95-49b9-b057-b5e5cfc59182 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:31.633556757 +0000 UTC m=+10.088270364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret") pod "global-pull-secret-syncer-c8jkn" (UID: "2b0d8426-5e95-49b9-b057-b5e5cfc59182") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:27.734527 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:27.733878    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:27.734527 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.734108    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:04:27.734527 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.734129    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:04:27.734527 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.734142    2560 projected.go:196] Error preparing data for projected volume kube-api-access-dnh2n for pod openshift-network-diagnostics/network-check-target-8ckgj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:27.734527 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:27.734198    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n podName:35ef3246-ba7b-44fd-8a27-d59603445a9a nodeName:}" failed. No retries permitted until 2026-07-22 15:04:31.734179843 +0000 UTC m=+10.188893441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnh2n" (UniqueName: "kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n") pod "network-check-target-8ckgj" (UID: "35ef3246-ba7b-44fd-8a27-d59603445a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:29.596720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:29.595870    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:29.596720 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:29.596016    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:29.596720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:29.596429    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:29.596720 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:29.596555    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:29.596720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:29.596611    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:29.596720 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:29.596677    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:31.596368 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:31.596188    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:31.596368 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:31.596222    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:31.596368 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:31.596226    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:31.596368 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.596336    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:31.596977 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.596492    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:31.596977 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.596563    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:31.668664 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:31.668628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:31.668844 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:31.668693    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:31.668844 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.668837    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:31.668957 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.668903    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs podName:ba27779c-af36-4803-88e3-0f0f25a01210 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:39.668882011 +0000 UTC m=+18.123595606 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs") pod "network-metrics-daemon-w9tkp" (UID: "ba27779c-af36-4803-88e3-0f0f25a01210") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:31.669021 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.668971    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:31.669021 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.669005    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret podName:2b0d8426-5e95-49b9-b057-b5e5cfc59182 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:39.668993804 +0000 UTC m=+18.123707397 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret") pod "global-pull-secret-syncer-c8jkn" (UID: "2b0d8426-5e95-49b9-b057-b5e5cfc59182") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:31.769183 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:31.769143    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:31.769404 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.769360    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:04:31.769404 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.769386    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:04:31.769404 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.769401    2560 projected.go:196] Error preparing data for projected volume kube-api-access-dnh2n for pod openshift-network-diagnostics/network-check-target-8ckgj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:31.769589 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:31.769479    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n podName:35ef3246-ba7b-44fd-8a27-d59603445a9a nodeName:}" failed. No retries permitted until 2026-07-22 15:04:39.769458297 +0000 UTC m=+18.224171911 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnh2n" (UniqueName: "kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n") pod "network-check-target-8ckgj" (UID: "35ef3246-ba7b-44fd-8a27-d59603445a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:33.596396 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:33.596359    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:33.596886 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:33.596410    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:33.596886 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:33.596359    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:33.596886 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:33.596536    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:33.597049 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:33.597012    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:33.597162 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:33.597094    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:35.596714 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:35.596681    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:35.596714 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:35.596701    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:35.597207 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:35.596689    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:35.597207 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:35.596800    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:35.597207 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:35.596910    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:35.597207 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:35.596981    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:37.596521 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:37.596492    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:37.596990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:37.596492    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:37.596990 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:37.596596    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:37.596990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:37.596492    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:37.596990 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:37.596727    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:37.596990 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:37.596806    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:39.596119 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:39.596080    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:39.596589 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:39.596080    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:39.596589 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.596333    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:39.596589 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.596205    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:39.596589 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:39.596095    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:39.596589 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.596438    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:39.728262 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:39.728235    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:39.728418 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:39.728285    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:39.728418 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.728379    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:39.728418 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.728382    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:39.728537 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.728425    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret podName:2b0d8426-5e95-49b9-b057-b5e5cfc59182 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:55.728411653 +0000 UTC m=+34.183125246 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret") pod "global-pull-secret-syncer-c8jkn" (UID: "2b0d8426-5e95-49b9-b057-b5e5cfc59182") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:39.728537 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.728438    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs podName:ba27779c-af36-4803-88e3-0f0f25a01210 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:55.728431497 +0000 UTC m=+34.183145090 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs") pod "network-metrics-daemon-w9tkp" (UID: "ba27779c-af36-4803-88e3-0f0f25a01210") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:39.829436 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:39.829407    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:39.829572 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.829557    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:04:39.829618 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.829574    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:04:39.829618 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.829585    2560 projected.go:196] Error preparing data for projected volume kube-api-access-dnh2n for pod openshift-network-diagnostics/network-check-target-8ckgj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:39.829682 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:39.829632    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n podName:35ef3246-ba7b-44fd-8a27-d59603445a9a nodeName:}" failed. No retries permitted until 2026-07-22 15:04:55.829619386 +0000 UTC m=+34.284332979 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnh2n" (UniqueName: "kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n") pod "network-check-target-8ckgj" (UID: "35ef3246-ba7b-44fd-8a27-d59603445a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:41.596457 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:41.596419    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:41.596457 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:41.596436    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:41.596995 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:41.596490    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:41.596995 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:41.596649    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:41.596995 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:41.596791    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:41.596995 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:41.596885    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:43.596481 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:43.596437    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:43.596481 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:43.596437    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:43.596951 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:43.596457    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:43.596951 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:43.596550    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:43.596951 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:43.596661    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:43.596951 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:43.596743    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:45.596244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.596073    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:45.596599 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.596134    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:45.596599 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:45.596331    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:45.596599 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.596134    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:45.596599 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:45.596392    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:45.596599 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:45.596514    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:45.733393 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.733198    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" event={"ID":"ddc4a6bf9a8830c582e31a32617937e6","Type":"ContainerStarted","Data":"7ee375b7724bdddf4f20c9c3e4b2e138855189a3beb65fbcb35db6df88de9800"}
Jul 22 15:04:45.734668 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.734643    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-45g86" event={"ID":"871c5594-a183-4116-8d8a-20ee028153f5","Type":"ContainerStarted","Data":"6c2a9dd2308e1d8cb07328f9939fa3a8aa7254cd815e4c5fac69c6a8eb84c019"}
Jul 22 15:04:45.737360 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.737340    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:04:45.737672 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.737654    2560 generic.go:356] "Generic (PLEG): container finished" podID="9d0d4357-a4b6-43fb-8587-4b50695f40e5" containerID="db6370779255be3bc531da5a70d681febb2ffe3c3784633d8a02afdebcdf1308" exitCode=1
Jul 22 15:04:45.737745 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.737724    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerStarted","Data":"a0a82919905d8c3037b9f01fabadc8c330425fb495f25db4c7264e83815415b3"}
Jul 22 15:04:45.737817 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.737804    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerStarted","Data":"28f3e15c960b5e470fc64d7fba94a8d05e7543b3b8173eed5ee064ce2932668f"}
Jul 22 15:04:45.737882 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.737868    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerStarted","Data":"f4d7d67196fb4dc18bd365aeb05341e4554e9c2345987a6d07a8a4a46c744d38"}
Jul 22 15:04:45.737936 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.737920    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerStarted","Data":"8ea6a4389aa4e1b74fb9b7b1ab43234e446a0cedfeffb95126dd488ad09a9b2c"}
Jul 22 15:04:45.737989 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.737967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerDied","Data":"db6370779255be3bc531da5a70d681febb2ffe3c3784633d8a02afdebcdf1308"}
Jul 22 15:04:45.738058 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.738043    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerStarted","Data":"3d833f52064eedd973c5cbd34933f304e22e8ccacfe1b3f7b24d931a595790e0"}
Jul 22 15:04:45.739146 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.739124    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bjq6l" event={"ID":"aa399725-34cb-4e9a-b711-97b1178f7915","Type":"ContainerStarted","Data":"607bc7bb817ef50db5a31090507ebc39005a1b174851f8cb5de8d3e8eedf7d6c"}
Jul 22 15:04:45.750556 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.750517    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-80.ec2.internal" podStartSLOduration=22.750504316 podStartE2EDuration="22.750504316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:45.750398247 +0000 UTC m=+24.205111863" watchObservedRunningTime="2026-07-22 15:04:45.750504316 +0000 UTC m=+24.205217930"
Jul 22 15:04:45.776129 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.776095    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bjq6l" podStartSLOduration=4.029013602 podStartE2EDuration="23.77608315s" totalImagesPullingTime="19.747069548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:45.775989305 +0000 UTC m=+24.230702920" watchObservedRunningTime="2026-07-22 15:04:45.77608315 +0000 UTC m=+24.230796764"
Jul 22 15:04:45.801001 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:45.800962    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-45g86" podStartSLOduration=3.65347402 podStartE2EDuration="23.800947618s" totalImagesPullingTime="20.147473598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:45.800697716 +0000 UTC m=+24.255411331" watchObservedRunningTime="2026-07-22 15:04:45.800947618 +0000 UTC m=+24.255661232"
Jul 22 15:04:46.742810 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.742775    2560 generic.go:356] "Generic (PLEG): container finished" podID="b1e56680-b7ed-4958-848e-7c7e0e75c826" containerID="33459278eb75302d29b0b1e0dd3599427f27854317f7980f3bada5956c95e3d6" exitCode=0
Jul 22 15:04:46.743770 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.742871    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8g44" event={"ID":"b1e56680-b7ed-4958-848e-7c7e0e75c826","Type":"ContainerDied","Data":"33459278eb75302d29b0b1e0dd3599427f27854317f7980f3bada5956c95e3d6"}
Jul 22 15:04:46.744659 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.744479    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dt7cb" event={"ID":"e608c87c-656a-470f-9bb6-d2125d19fd29","Type":"ContainerStarted","Data":"dc7de335defd98bb8c8cf2bd6cfd2abc5f2fd310b5b81cb346afce05916ce3dc"}
Jul 22 15:04:46.746244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.746220    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6" event={"ID":"538d0762-b726-4179-83e0-e5028e629c65","Type":"ContainerStarted","Data":"f4f34be3ded5e7d3af519e09b32bd5bc5e86f8c66fa30723ae80fff1b2efb045"}
Jul 22 15:04:46.747825 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.747774    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hkpl4" event={"ID":"3d1e6293-7310-4977-81ce-f804acdcbcd0","Type":"ContainerStarted","Data":"bd0cd3240905188aae068edf496890540ef418fabfce8a2a2c0b445110373963"}
Jul 22 15:04:46.749379 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.749355    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z4765" event={"ID":"dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f","Type":"ContainerStarted","Data":"c1bd336ab3e8466ccc8bd2ab1d8c5c7e74d3979e88427dd8cce5a2e9c26c1805"}
Jul 22 15:04:46.750943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.750919    2560 generic.go:356] "Generic (PLEG): container finished" podID="35e0fc6a14943475efe90a14e95086ef" containerID="4b24fdace40a2c88dc0d44ebd77b76dcca32c443fea9d5cba536c5946eb67aeb" exitCode=0
Jul 22 15:04:46.751031 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.751004    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" event={"ID":"35e0fc6a14943475efe90a14e95086ef","Type":"ContainerDied","Data":"4b24fdace40a2c88dc0d44ebd77b76dcca32c443fea9d5cba536c5946eb67aeb"}
Jul 22 15:04:46.752490 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.752467    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4nbnn" event={"ID":"12e87e9c-0d5e-4be3-99be-110478935005","Type":"ContainerStarted","Data":"5e8a19b69d693a468cf9f255f7b348200372e6d97ce02d5538010be7e03bfb43"}
Jul 22 15:04:46.782547 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.782496    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4nbnn" podStartSLOduration=4.894399799 podStartE2EDuration="24.782438543s" totalImagesPullingTime="19.888038744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:46.782007165 +0000 UTC m=+25.236720781" watchObservedRunningTime="2026-07-22 15:04:46.782438543 +0000 UTC m=+25.237152157"
Jul 22 15:04:46.815140 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.814850    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z4765" podStartSLOduration=4.69284193 podStartE2EDuration="24.814828324s" totalImagesPullingTime="20.121986394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:46.814640885 +0000 UTC m=+25.269354501" watchObservedRunningTime="2026-07-22 15:04:46.814828324 +0000 UTC m=+25.269541944"
Jul 22 15:04:46.829910 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.829770    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hkpl4" podStartSLOduration=4.761257053 podStartE2EDuration="24.829750407s" totalImagesPullingTime="20.068493354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:46.829326048 +0000 UTC m=+25.284039656" watchObservedRunningTime="2026-07-22 15:04:46.829750407 +0000 UTC m=+25.284464021"
Jul 22 15:04:46.846287 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:46.846228    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dt7cb" podStartSLOduration=4.7971606730000005 podStartE2EDuration="24.846207382s" totalImagesPullingTime="20.049046709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:46.845376442 +0000 UTC m=+25.300090056" watchObservedRunningTime="2026-07-22 15:04:46.846207382 +0000 UTC m=+25.300920998"
Jul 22 15:04:47.439777 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.439753    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 15:04:47.596362 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.596280    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:47.596549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.596390    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:47.596549 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:47.596401    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:47.596549 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:47.596510    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:47.596549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.596400    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:47.596768 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:47.596616    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:47.707956 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.707921    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:47.708669 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.708641    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:47.756274 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.756241    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6" event={"ID":"538d0762-b726-4179-83e0-e5028e629c65","Type":"ContainerStarted","Data":"e691b31e413203d21bfd57184fa88ba0230558ab86810a51f739cdf8afb3b717"}
Jul 22 15:04:47.758063 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.758000    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" event={"ID":"35e0fc6a14943475efe90a14e95086ef","Type":"ContainerStarted","Data":"62f66775e0287f4588013e9b1c8c8f29d57f2acb1fb222b24afcb2e7859ffcc0"}
Jul 22 15:04:47.758337 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.758321    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:47.758868 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.758848    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4nbnn"
Jul 22 15:04:47.775428 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:47.775392    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-80.ec2.internal" podStartSLOduration=24.775376942 podStartE2EDuration="24.775376942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:47.775315071 +0000 UTC m=+26.230028686" watchObservedRunningTime="2026-07-22 15:04:47.775376942 +0000 UTC m=+26.230090556"
Jul 22 15:04:48.138167 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:48.138037    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T15:04:47.43977356Z","UUID":"4e854a63-d317-4414-9ce8-4dc89df2b8aa","Handler":null,"Name":"","Endpoint":""}
Jul 22 15:04:48.139795 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:48.139775    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 15:04:48.139930 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:48.139825    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 15:04:48.763669 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:48.763596    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:04:48.764084 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:48.763992    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerStarted","Data":"6665d2b362fa7754f09fc8b01f3734889ce4211d482e1eca01c20230f196433c"}
Jul 22 15:04:48.766274 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:48.766246    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6" event={"ID":"538d0762-b726-4179-83e0-e5028e629c65","Type":"ContainerStarted","Data":"f606ba9fbf0304c40cf8b2fc5763282b1ce8cffab220a55f27e856c640a99e31"}
Jul 22 15:04:49.596319 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:49.596285    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:49.596492 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:49.596296    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:49.596492 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:49.596431    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:49.596492 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:49.596469    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:49.596492 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:49.596298    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:49.596633 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:49.596534    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:51.596607 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.596394    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:51.597263 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.596435    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:51.597263 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:51.596696    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:51.597263 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:51.596772    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:51.597263 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.596442    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:51.597263 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:51.596868    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:51.777082 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.777005    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:04:51.777415 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.777393    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerStarted","Data":"25555675dd338e0af9a17dd8de39f37eda30d133f761041e4c75772d3368fa10"}
Jul 22 15:04:51.777654 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.777628    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:51.777875 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.777859    2560 scope.go:117] "RemoveContainer" containerID="db6370779255be3bc531da5a70d681febb2ffe3c3784633d8a02afdebcdf1308"
Jul 22 15:04:51.779362 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.779336    2560 generic.go:356] "Generic (PLEG): container finished" podID="b1e56680-b7ed-4958-848e-7c7e0e75c826" containerID="d438d949658501a998db2bee560fa9defc841713855b616d83f90e2d6af87c0e" exitCode=0
Jul 22 15:04:51.779481 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.779378    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8g44" event={"ID":"b1e56680-b7ed-4958-848e-7c7e0e75c826","Type":"ContainerDied","Data":"d438d949658501a998db2bee560fa9defc841713855b616d83f90e2d6af87c0e"}
Jul 22 15:04:51.793643 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.793620    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:51.823592 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:51.823546    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sbgt6" podStartSLOduration=7.495722085 podStartE2EDuration="29.823532081s" totalImagesPullingTime="22.327809996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:48.803227642 +0000 UTC m=+27.257941259" watchObservedRunningTime="2026-07-22 15:04:51.823532081 +0000 UTC m=+30.278245717"
Jul 22 15:04:52.785066 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:52.785039    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:04:52.785716 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:52.785470    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" event={"ID":"9d0d4357-a4b6-43fb-8587-4b50695f40e5","Type":"ContainerStarted","Data":"e6aa3d6da130b2a567d6683314c9fafe9c33ed23e3308c12a336bbfdad2601ef"}
Jul 22 15:04:52.785716 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:52.785634    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:52.785716 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:52.785693    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:52.800303 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:52.800279    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:04:52.828000 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:52.827958    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wf658" podStartSLOduration=10.667813938 podStartE2EDuration="30.827946025s" totalImagesPullingTime="20.160132087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:04:52.826722027 +0000 UTC m=+31.281435655" watchObservedRunningTime="2026-07-22 15:04:52.827946025 +0000 UTC m=+31.282659639"
Jul 22 15:04:53.556867 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:53.556830    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w9tkp"]
Jul 22 15:04:53.557032 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:53.556975    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:53.557085 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:53.557065    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:53.559626 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:53.559598    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c8jkn"]
Jul 22 15:04:53.559753 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:53.559724    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:53.559849 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:53.559830    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:53.575736 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:53.575699    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ckgj"]
Jul 22 15:04:53.575912 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:53.575838    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:53.575977 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:53.575943    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:55.596394 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:55.596326    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:55.596394 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:55.596350    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:55.596830 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:55.596326    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:55.596830 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.596415    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:55.596830 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.596518    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:55.596830 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.596583    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:55.736695 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:55.736665    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:55.736825 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:55.736724    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:55.736825 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.736802    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:55.736825 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.736812    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:55.736935 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.736876    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs podName:ba27779c-af36-4803-88e3-0f0f25a01210 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:27.736847466 +0000 UTC m=+66.191561093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs") pod "network-metrics-daemon-w9tkp" (UID: "ba27779c-af36-4803-88e3-0f0f25a01210") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 15:04:55.736935 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.736892    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret podName:2b0d8426-5e95-49b9-b057-b5e5cfc59182 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:27.736884946 +0000 UTC m=+66.191598589 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret") pod "global-pull-secret-syncer-c8jkn" (UID: "2b0d8426-5e95-49b9-b057-b5e5cfc59182") : object "kube-system"/"original-pull-secret" not registered
Jul 22 15:04:55.794815 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:55.794785    2560 generic.go:356] "Generic (PLEG): container finished" podID="b1e56680-b7ed-4958-848e-7c7e0e75c826" containerID="78fa9965e614af7cf47c693cfba8a79a2cf8b4761737f0b2aba06aa0d0112141" exitCode=0
Jul 22 15:04:55.794968 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:55.794853    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8g44" event={"ID":"b1e56680-b7ed-4958-848e-7c7e0e75c826","Type":"ContainerDied","Data":"78fa9965e614af7cf47c693cfba8a79a2cf8b4761737f0b2aba06aa0d0112141"}
Jul 22 15:04:55.837066 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:55.837028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:55.837199 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.837141    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 15:04:55.837199 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.837158    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 15:04:55.837199 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.837167    2560 projected.go:196] Error preparing data for projected volume kube-api-access-dnh2n for pod openshift-network-diagnostics/network-check-target-8ckgj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:55.837314 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:55.837211    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n podName:35ef3246-ba7b-44fd-8a27-d59603445a9a nodeName:}" failed. No retries permitted until 2026-07-22 15:05:27.837198167 +0000 UTC m=+66.291911774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dnh2n" (UniqueName: "kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n") pod "network-check-target-8ckgj" (UID: "35ef3246-ba7b-44fd-8a27-d59603445a9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 15:04:57.596770 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:57.596721    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:57.596770 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:57.596772    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:57.597195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:57.596731    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:57.597195 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:57.596839    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9tkp" podUID="ba27779c-af36-4803-88e3-0f0f25a01210"
Jul 22 15:04:57.597195 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:57.596938    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8jkn" podUID="2b0d8426-5e95-49b9-b057-b5e5cfc59182"
Jul 22 15:04:57.597195 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:57.596992    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ckgj" podUID="35ef3246-ba7b-44fd-8a27-d59603445a9a"
Jul 22 15:04:57.802397 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:57.802365    2560 generic.go:356] "Generic (PLEG): container finished" podID="b1e56680-b7ed-4958-848e-7c7e0e75c826" containerID="e3881cb2798e78eaf9f999471ced9e78c856f9daa9de3bf4f931c4df181e9b1e" exitCode=0
Jul 22 15:04:57.802576 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:57.802425    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8g44" event={"ID":"b1e56680-b7ed-4958-848e-7c7e0e75c826","Type":"ContainerDied","Data":"e3881cb2798e78eaf9f999471ced9e78c856f9daa9de3bf4f931c4df181e9b1e"}
Jul 22 15:04:58.314377 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.314348    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-80.ec2.internal" event="NodeReady"
Jul 22 15:04:58.314569 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.314497    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 15:04:58.377286 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.377190    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56df5f6fcb-jvxb8"]
Jul 22 15:04:58.379130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.379110    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.383064 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.383046    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 15:04:58.383159 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.383094    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xgdsc\""
Jul 22 15:04:58.383359 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.383338    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 15:04:58.383359 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.383354    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 15:04:58.393169 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.393148    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 15:04:58.405247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.405225    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56df5f6fcb-jvxb8"]
Jul 22 15:04:58.416470 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.416428    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pblnf"]
Jul 22 15:04:58.418619 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.418602    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gmpdw"]
Jul 22 15:04:58.418751 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.418736    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.420794 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.420778    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gmpdw"
Jul 22 15:04:58.421983 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.421968    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 15:04:58.422287 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.422270    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 15:04:58.422343 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.422282    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s4xt5\""
Jul 22 15:04:58.424158 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.424135    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 15:04:58.424259 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.424166    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gqg9k\""
Jul 22 15:04:58.424259 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.424135    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 15:04:58.424259 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.424202    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 15:04:58.436721 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.436703    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pblnf"]
Jul 22 15:04:58.451853 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.451830    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gmpdw"]
Jul 22 15:04:58.457367 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.457343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjjj5\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-kube-api-access-hjjj5\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.457418 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.457387    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b9720254-d650-4e81-9417-3355eb50cce1-ca-trust-extracted\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.457418 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.457408    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-trusted-ca\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.457553 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.457536    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.457601 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.457561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-installation-pull-secrets\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.457652 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.457629    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-bound-sa-token\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.457702 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.457672    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-image-registry-private-configuration\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.457762 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.457720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-registry-certificates\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.558085 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558047    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c583dee-6f25-4c83-9858-da1763f2a5f5-cert\") pod \"ingress-canary-gmpdw\" (UID: \"6c583dee-6f25-4c83-9858-da1763f2a5f5\") " pod="openshift-ingress-canary/ingress-canary-gmpdw"
Jul 22 15:04:58.558085 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxkbt\" (UniqueName: \"kubernetes.io/projected/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-kube-api-access-rxkbt\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558104    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-config-volume\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558140    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjjj5\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-kube-api-access-hjjj5\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558157    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b9720254-d650-4e81-9417-3355eb50cce1-ca-trust-extracted\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558174    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-trusted-ca\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558199    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558213    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-installation-pull-secrets\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558233    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t252r\" (UniqueName: \"kubernetes.io/projected/6c583dee-6f25-4c83-9858-da1763f2a5f5-kube-api-access-t252r\") pod \"ingress-canary-gmpdw\" (UID: \"6c583dee-6f25-4c83-9858-da1763f2a5f5\") " pod="openshift-ingress-canary/ingress-canary-gmpdw"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-bound-sa-token\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558278    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-image-registry-private-configuration\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.558325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-tmp-dir\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.558893 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:58.558330    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 15:04:58.558893 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:58.558364    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56df5f6fcb-jvxb8: secret "image-registry-tls" not found
Jul 22 15:04:58.558893 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:58.558425    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls podName:b9720254-d650-4e81-9417-3355eb50cce1 nodeName:}" failed. No retries permitted until 2026-07-22 15:04:59.058405265 +0000 UTC m=+37.513118865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls") pod "image-registry-56df5f6fcb-jvxb8" (UID: "b9720254-d650-4e81-9417-3355eb50cce1") : secret "image-registry-tls" not found
Jul 22 15:04:58.558893 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558346    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-registry-certificates\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.558893 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.558811    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b9720254-d650-4e81-9417-3355eb50cce1-ca-trust-extracted\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.559261 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.559090    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-registry-certificates\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.559316 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.559307    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-trusted-ca\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.562353 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.562330    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-image-registry-private-configuration\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.562353 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.562338    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-installation-pull-secrets\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.573825 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.573754    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-bound-sa-token\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.573933 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.573915    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjjj5\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-kube-api-access-hjjj5\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:58.659805 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.659759    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c583dee-6f25-4c83-9858-da1763f2a5f5-cert\") pod \"ingress-canary-gmpdw\" (UID: \"6c583dee-6f25-4c83-9858-da1763f2a5f5\") " pod="openshift-ingress-canary/ingress-canary-gmpdw"
Jul 22 15:04:58.659805 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.659807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxkbt\" (UniqueName: \"kubernetes.io/projected/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-kube-api-access-rxkbt\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.659805 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.659827    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.660413 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.659847    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-config-volume\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.660413 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:58.659927    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 15:04:58.660413 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:58.659996    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls podName:08cc6e66-2c7b-483d-9d70-b96de5dd2c7a nodeName:}" failed. No retries permitted until 2026-07-22 15:04:59.159979818 +0000 UTC m=+37.614693444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls") pod "dns-default-pblnf" (UID: "08cc6e66-2c7b-483d-9d70-b96de5dd2c7a") : secret "dns-default-metrics-tls" not found
Jul 22 15:04:58.660413 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.660027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t252r\" (UniqueName: \"kubernetes.io/projected/6c583dee-6f25-4c83-9858-da1763f2a5f5-kube-api-access-t252r\") pod \"ingress-canary-gmpdw\" (UID: \"6c583dee-6f25-4c83-9858-da1763f2a5f5\") " pod="openshift-ingress-canary/ingress-canary-gmpdw"
Jul 22 15:04:58.660413 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.660062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-tmp-dir\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.660413 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.660319    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-tmp-dir\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.660636 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.660439    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-config-volume\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.662203 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.662173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c583dee-6f25-4c83-9858-da1763f2a5f5-cert\") pod \"ingress-canary-gmpdw\" (UID: \"6c583dee-6f25-4c83-9858-da1763f2a5f5\") " pod="openshift-ingress-canary/ingress-canary-gmpdw"
Jul 22 15:04:58.685147 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.685123    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxkbt\" (UniqueName: \"kubernetes.io/projected/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-kube-api-access-rxkbt\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:58.685834 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.685800    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t252r\" (UniqueName: \"kubernetes.io/projected/6c583dee-6f25-4c83-9858-da1763f2a5f5-kube-api-access-t252r\") pod \"ingress-canary-gmpdw\" (UID: \"6c583dee-6f25-4c83-9858-da1763f2a5f5\") " pod="openshift-ingress-canary/ingress-canary-gmpdw"
Jul 22 15:04:58.732876 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.732847    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gmpdw"
Jul 22 15:04:58.898808 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:58.898772    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gmpdw"]
Jul 22 15:04:58.903032 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:04:58.903005    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c583dee_6f25_4c83_9858_da1763f2a5f5.slice/crio-1c74b4e89687983fb6d95e16ea697fb66b34b049e8bd66b1d4d3ac8d59cc1f3e WatchSource:0}: Error finding container 1c74b4e89687983fb6d95e16ea697fb66b34b049e8bd66b1d4d3ac8d59cc1f3e: Status 404 returned error can't find the container with id 1c74b4e89687983fb6d95e16ea697fb66b34b049e8bd66b1d4d3ac8d59cc1f3e
Jul 22 15:04:59.063873 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.063825    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:04:59.064059 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:59.063989    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 15:04:59.064059 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:59.064009    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56df5f6fcb-jvxb8: secret "image-registry-tls" not found
Jul 22 15:04:59.064125 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:59.064076    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls podName:b9720254-d650-4e81-9417-3355eb50cce1 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:00.064060108 +0000 UTC m=+38.518773701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls") pod "image-registry-56df5f6fcb-jvxb8" (UID: "b9720254-d650-4e81-9417-3355eb50cce1") : secret "image-registry-tls" not found
Jul 22 15:04:59.164782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.164677    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:04:59.164918 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:59.164839    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 15:04:59.164918 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:04:59.164910    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls podName:08cc6e66-2c7b-483d-9d70-b96de5dd2c7a nodeName:}" failed. No retries permitted until 2026-07-22 15:05:00.164894619 +0000 UTC m=+38.619608212 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls") pod "dns-default-pblnf" (UID: "08cc6e66-2c7b-483d-9d70-b96de5dd2c7a") : secret "dns-default-metrics-tls" not found
Jul 22 15:04:59.596461 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.596372    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:04:59.596461 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.596411    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:04:59.596461 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.596413    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:04:59.600164 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.600142    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 15:04:59.602280 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.602209    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 15:04:59.602280 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.602225    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 15:04:59.602280 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.602259    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 15:04:59.602280 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.602279    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9ldrl\""
Jul 22 15:04:59.602565 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.602280    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zbjvd\""
Jul 22 15:04:59.809366 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:04:59.809330    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gmpdw" event={"ID":"6c583dee-6f25-4c83-9858-da1763f2a5f5","Type":"ContainerStarted","Data":"1c74b4e89687983fb6d95e16ea697fb66b34b049e8bd66b1d4d3ac8d59cc1f3e"}
Jul 22 15:05:00.071809 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:00.071771    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:05:00.071993 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:00.071967    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 15:05:00.072058 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:00.071998    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56df5f6fcb-jvxb8: secret "image-registry-tls" not found
Jul 22 15:05:00.072100 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:00.072065    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls podName:b9720254-d650-4e81-9417-3355eb50cce1 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:02.072048766 +0000 UTC m=+40.526762377 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls") pod "image-registry-56df5f6fcb-jvxb8" (UID: "b9720254-d650-4e81-9417-3355eb50cce1") : secret "image-registry-tls" not found
Jul 22 15:05:00.172828 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:00.172788    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:05:00.173045 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:00.173023    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 15:05:00.173113 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:00.173094    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls podName:08cc6e66-2c7b-483d-9d70-b96de5dd2c7a nodeName:}" failed. No retries permitted until 2026-07-22 15:05:02.173073941 +0000 UTC m=+40.627787534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls") pod "dns-default-pblnf" (UID: "08cc6e66-2c7b-483d-9d70-b96de5dd2c7a") : secret "dns-default-metrics-tls" not found
Jul 22 15:05:02.087676 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:02.087629    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:05:02.088086 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:02.087810    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 15:05:02.088086 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:02.087836    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56df5f6fcb-jvxb8: secret "image-registry-tls" not found
Jul 22 15:05:02.088086 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:02.087907    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls podName:b9720254-d650-4e81-9417-3355eb50cce1 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:06.087885365 +0000 UTC m=+44.542598975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls") pod "image-registry-56df5f6fcb-jvxb8" (UID: "b9720254-d650-4e81-9417-3355eb50cce1") : secret "image-registry-tls" not found
Jul 22 15:05:02.188944 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:02.188785    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:05:02.188944 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:02.188942    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 15:05:02.189120 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:02.189006    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls podName:08cc6e66-2c7b-483d-9d70-b96de5dd2c7a nodeName:}" failed. No retries permitted until 2026-07-22 15:05:06.188986256 +0000 UTC m=+44.643699877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls") pod "dns-default-pblnf" (UID: "08cc6e66-2c7b-483d-9d70-b96de5dd2c7a") : secret "dns-default-metrics-tls" not found
Jul 22 15:05:02.816999 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:02.816967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gmpdw" event={"ID":"6c583dee-6f25-4c83-9858-da1763f2a5f5","Type":"ContainerStarted","Data":"d3f83b4cb0ba424decec001f391b2c55491bd108b8421fb6932f0c3bc175d5db"}
Jul 22 15:05:02.840124 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:02.840067    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gmpdw" podStartSLOduration=1.597881898 podStartE2EDuration="4.84004723s" totalImagesPullingTime="3.242165332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:04:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:02.839479785 +0000 UTC m=+41.294193399" watchObservedRunningTime="2026-07-22 15:05:02.84004723 +0000 UTC m=+41.294760848"
Jul 22 15:05:06.023171 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.023133    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"]
Jul 22 15:05:06.045576 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.045551    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"]
Jul 22 15:05:06.045687 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.045647    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:06.049260 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.049236    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rs8z7\""
Jul 22 15:05:06.050277 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.050259    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 22 15:05:06.050382 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.050306    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 22 15:05:06.050382 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.050350    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 22 15:05:06.122903 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.122883    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:05:06.123042 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.123027    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 15:05:06.123080 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.123044    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56df5f6fcb-jvxb8: secret "image-registry-tls" not found
Jul 22 15:05:06.123109 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.123101    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls podName:b9720254-d650-4e81-9417-3355eb50cce1 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:14.123083345 +0000 UTC m=+52.577796937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls") pod "image-registry-56df5f6fcb-jvxb8" (UID: "b9720254-d650-4e81-9417-3355eb50cce1") : secret "image-registry-tls" not found
Jul 22 15:05:06.223700 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.223668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wr8f\" (UniqueName: \"kubernetes.io/projected/df999d2c-1dd2-44d4-8731-63ec81feb9fb-kube-api-access-5wr8f\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:06.223802 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.223709    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:05:06.223802 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.223740    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:06.223889 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.223868    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 15:05:06.223958 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.223946    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls podName:08cc6e66-2c7b-483d-9d70-b96de5dd2c7a nodeName:}" failed. No retries permitted until 2026-07-22 15:05:14.22392323 +0000 UTC m=+52.678636897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls") pod "dns-default-pblnf" (UID: "08cc6e66-2c7b-483d-9d70-b96de5dd2c7a") : secret "dns-default-metrics-tls" not found
Jul 22 15:05:06.249070 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.249044    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"]
Jul 22 15:05:06.265040 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.265016    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.268489 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.268469    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 22 15:05:06.269532 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.269278    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 22 15:05:06.269532 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.269294    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 22 15:05:06.269532 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.269282    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 22 15:05:06.269708 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.269563    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fkdqq\""
Jul 22 15:05:06.287184 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.287094    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"]
Jul 22 15:05:06.324321 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.324299    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wr8f\" (UniqueName: \"kubernetes.io/projected/df999d2c-1dd2-44d4-8731-63ec81feb9fb-kube-api-access-5wr8f\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:06.324427 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.324342    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:06.324493 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.324480    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 15:05:06.324548 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.324537    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls podName:df999d2c-1dd2-44d4-8731-63ec81feb9fb nodeName:}" failed. No retries permitted until 2026-07-22 15:05:06.824522958 +0000 UTC m=+45.279236551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-j5b5d" (UID: "df999d2c-1dd2-44d4-8731-63ec81feb9fb") : secret "samples-operator-tls" not found
Jul 22 15:05:06.349234 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.349207    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wr8f\" (UniqueName: \"kubernetes.io/projected/df999d2c-1dd2-44d4-8731-63ec81feb9fb-kube-api-access-5wr8f\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:06.358015 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.357989    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-d789dc46-blhfw"]
Jul 22 15:05:06.378985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.378966    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-mnzqv"]
Jul 22 15:05:06.379137 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.379108    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.382990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.382965    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g4j4d\""
Jul 22 15:05:06.383394 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.383374    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 15:05:06.383628 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.383614    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 22 15:05:06.383879 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.383861    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 22 15:05:06.384125 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.384105    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 22 15:05:06.384408 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.384394    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 15:05:06.384472 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.384437    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 22 15:05:06.399177 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.399145    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-mnzqv"]
Jul 22 15:05:06.399177 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.399179    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-d789dc46-blhfw"]
Jul 22 15:05:06.399355 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.399258    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.402737 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.402716    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bj8l5\""
Jul 22 15:05:06.402859 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.402744    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 22 15:05:06.403003 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.402990    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 22 15:05:06.403386 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.403370    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 22 15:05:06.403497 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.403490    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 22 15:05:06.409378 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.409360    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 22 15:05:06.424981 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.424958    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/880ce076-2799-46e2-b8d2-950e9f74dee0-config\") pod \"service-ca-operator-5bcbc94c78-4qt77\" (UID: \"880ce076-2799-46e2-b8d2-950e9f74dee0\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.425067 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.425024    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk54k\" (UniqueName: \"kubernetes.io/projected/880ce076-2799-46e2-b8d2-950e9f74dee0-kube-api-access-vk54k\") pod \"service-ca-operator-5bcbc94c78-4qt77\" (UID: \"880ce076-2799-46e2-b8d2-950e9f74dee0\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.425106 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.425095    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/880ce076-2799-46e2-b8d2-950e9f74dee0-serving-cert\") pod \"service-ca-operator-5bcbc94c78-4qt77\" (UID: \"880ce076-2799-46e2-b8d2-950e9f74dee0\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.526056 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526035    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/880ce076-2799-46e2-b8d2-950e9f74dee0-serving-cert\") pod \"service-ca-operator-5bcbc94c78-4qt77\" (UID: \"880ce076-2799-46e2-b8d2-950e9f74dee0\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.526195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526062    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.526195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526081    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8s98\" (UniqueName: \"kubernetes.io/projected/d1269636-75f4-4267-b9d6-302fd715c279-kube-api-access-j8s98\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.526195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-stats-auth\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.526195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526175    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/880ce076-2799-46e2-b8d2-950e9f74dee0-config\") pod \"service-ca-operator-5bcbc94c78-4qt77\" (UID: \"880ce076-2799-46e2-b8d2-950e9f74dee0\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.526349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526209    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-default-certificate\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.526349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526231    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlv7k\" (UniqueName: \"kubernetes.io/projected/9e0637ad-06fc-496c-8a90-884a39cf26d9-kube-api-access-wlv7k\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.526349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526272    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.526349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526314    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d1269636-75f4-4267-b9d6-302fd715c279-config\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.526349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d1269636-75f4-4267-b9d6-302fd715c279-serving-cert\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.526528 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526367    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1269636-75f4-4267-b9d6-302fd715c279-trusted-ca\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.526528 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk54k\" (UniqueName: \"kubernetes.io/projected/880ce076-2799-46e2-b8d2-950e9f74dee0-kube-api-access-vk54k\") pod \"service-ca-operator-5bcbc94c78-4qt77\" (UID: \"880ce076-2799-46e2-b8d2-950e9f74dee0\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.526736 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.526721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/880ce076-2799-46e2-b8d2-950e9f74dee0-config\") pod \"service-ca-operator-5bcbc94c78-4qt77\" (UID: \"880ce076-2799-46e2-b8d2-950e9f74dee0\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.528233 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.528217    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/880ce076-2799-46e2-b8d2-950e9f74dee0-serving-cert\") pod \"service-ca-operator-5bcbc94c78-4qt77\" (UID: \"880ce076-2799-46e2-b8d2-950e9f74dee0\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.538760 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.538707    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk54k\" (UniqueName: \"kubernetes.io/projected/880ce076-2799-46e2-b8d2-950e9f74dee0-kube-api-access-vk54k\") pod \"service-ca-operator-5bcbc94c78-4qt77\" (UID: \"880ce076-2799-46e2-b8d2-950e9f74dee0\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.575683 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.575661    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"
Jul 22 15:05:06.627306 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.627277    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d1269636-75f4-4267-b9d6-302fd715c279-config\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.627435 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.627333    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d1269636-75f4-4267-b9d6-302fd715c279-serving-cert\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.627435 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.627370    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1269636-75f4-4267-b9d6-302fd715c279-trusted-ca\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.627435 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.627398    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.627435 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.627416    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8s98\" (UniqueName: \"kubernetes.io/projected/d1269636-75f4-4267-b9d6-302fd715c279-kube-api-access-j8s98\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.627592 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.627464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-stats-auth\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.627592 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.627505    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-default-certificate\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.627592 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.627532    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlv7k\" (UniqueName: \"kubernetes.io/projected/9e0637ad-06fc-496c-8a90-884a39cf26d9-kube-api-access-wlv7k\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.627592 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.627545    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 15:05:06.627592 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.627590    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.627747 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.627625    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:07.127603771 +0000 UTC m=+45.582317387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : secret "router-metrics-certs-default" not found
Jul 22 15:05:06.627747 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.627683    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:07.127672774 +0000 UTC m=+45.582386384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : configmap references non-existent config key: service-ca.crt
Jul 22 15:05:06.628147 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.628092    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d1269636-75f4-4267-b9d6-302fd715c279-config\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.628441 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.628418    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d1269636-75f4-4267-b9d6-302fd715c279-trusted-ca\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.629801 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.629771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d1269636-75f4-4267-b9d6-302fd715c279-serving-cert\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.630007 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.629987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-default-certificate\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.630081 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.630042    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-stats-auth\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.638113 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.638092    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlv7k\" (UniqueName: \"kubernetes.io/projected/9e0637ad-06fc-496c-8a90-884a39cf26d9-kube-api-access-wlv7k\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:06.638229 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.638177    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8s98\" (UniqueName: \"kubernetes.io/projected/d1269636-75f4-4267-b9d6-302fd715c279-kube-api-access-j8s98\") pod \"console-operator-565b7b569d-mnzqv\" (UID: \"d1269636-75f4-4267-b9d6-302fd715c279\") " pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.707951 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.707917    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:06.729926 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.729892    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77"]
Jul 22 15:05:06.733748 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:06.733723    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod880ce076_2799_46e2_b8d2_950e9f74dee0.slice/crio-20ba9006299bc1ed5135bd91a56106d084bb348941e6b3204d8a4249084dc93a WatchSource:0}: Error finding container 20ba9006299bc1ed5135bd91a56106d084bb348941e6b3204d8a4249084dc93a: Status 404 returned error can't find the container with id 20ba9006299bc1ed5135bd91a56106d084bb348941e6b3204d8a4249084dc93a
Jul 22 15:05:06.828692 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.828661    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:06.828841 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.828813    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 15:05:06.828901 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:06.828887    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls podName:df999d2c-1dd2-44d4-8731-63ec81feb9fb nodeName:}" failed. No retries permitted until 2026-07-22 15:05:07.828866467 +0000 UTC m=+46.283580062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-j5b5d" (UID: "df999d2c-1dd2-44d4-8731-63ec81feb9fb") : secret "samples-operator-tls" not found
Jul 22 15:05:06.830226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.830200    2560 generic.go:356] "Generic (PLEG): container finished" podID="b1e56680-b7ed-4958-848e-7c7e0e75c826" containerID="71e3dc38f39a97ba4c9ed142c354a3e2ccd1e31c912d9c3b02e82f8cf826dd2b" exitCode=0
Jul 22 15:05:06.830362 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.830277    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8g44" event={"ID":"b1e56680-b7ed-4958-848e-7c7e0e75c826","Type":"ContainerDied","Data":"71e3dc38f39a97ba4c9ed142c354a3e2ccd1e31c912d9c3b02e82f8cf826dd2b"}
Jul 22 15:05:06.831679 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.831654    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77" event={"ID":"880ce076-2799-46e2-b8d2-950e9f74dee0","Type":"ContainerStarted","Data":"20ba9006299bc1ed5135bd91a56106d084bb348941e6b3204d8a4249084dc93a"}
Jul 22 15:05:06.850532 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:06.850512    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-mnzqv"]
Jul 22 15:05:06.853433 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:06.853404    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1269636_75f4_4267_b9d6_302fd715c279.slice/crio-19a97713348a7973e0209a2f9f6437d3e0171da3e1899d4ee9239737e9677cfe WatchSource:0}: Error finding container 19a97713348a7973e0209a2f9f6437d3e0171da3e1899d4ee9239737e9677cfe: Status 404 returned error can't find the container with id 19a97713348a7973e0209a2f9f6437d3e0171da3e1899d4ee9239737e9677cfe
Jul 22 15:05:07.131256 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:07.131200    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:07.131569 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:07.131263    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:07.131569 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:07.131353    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:08.131336438 +0000 UTC m=+46.586050031 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : configmap references non-existent config key: service-ca.crt
Jul 22 15:05:07.131569 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:07.131361    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 15:05:07.131569 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:07.131417    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:08.131401159 +0000 UTC m=+46.586114766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : secret "router-metrics-certs-default" not found
Jul 22 15:05:07.835366 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:07.835328    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv" event={"ID":"d1269636-75f4-4267-b9d6-302fd715c279","Type":"ContainerStarted","Data":"19a97713348a7973e0209a2f9f6437d3e0171da3e1899d4ee9239737e9677cfe"}
Jul 22 15:05:07.837144 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:07.837114    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:07.837296 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:07.837270    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 15:05:07.837356 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:07.837336    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls podName:df999d2c-1dd2-44d4-8731-63ec81feb9fb nodeName:}" failed. No retries permitted until 2026-07-22 15:05:09.837315956 +0000 UTC m=+48.292029569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-j5b5d" (UID: "df999d2c-1dd2-44d4-8731-63ec81feb9fb") : secret "samples-operator-tls" not found
Jul 22 15:05:07.838818 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:07.838789    2560 generic.go:356] "Generic (PLEG): container finished" podID="b1e56680-b7ed-4958-848e-7c7e0e75c826" containerID="f5c356afdd64d3d7552dd3ea236d82ad3291b8c0c063eb0f4a060cef7e5113de" exitCode=0
Jul 22 15:05:07.838920 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:07.838844    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8g44" event={"ID":"b1e56680-b7ed-4958-848e-7c7e0e75c826","Type":"ContainerDied","Data":"f5c356afdd64d3d7552dd3ea236d82ad3291b8c0c063eb0f4a060cef7e5113de"}
Jul 22 15:05:08.140558 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:08.140380    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:08.140558 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:08.140481    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:08.141306 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:08.140624    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 15:05:08.141306 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:08.140649    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:10.140629225 +0000 UTC m=+48.595342821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : configmap references non-existent config key: service-ca.crt
Jul 22 15:05:08.141306 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:08.140686    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:10.140669649 +0000 UTC m=+48.595383258 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : secret "router-metrics-certs-default" not found
Jul 22 15:05:08.845640 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:08.845599    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d8g44" event={"ID":"b1e56680-b7ed-4958-848e-7c7e0e75c826","Type":"ContainerStarted","Data":"ba53366de04bd2e6052581b3bec97ae97a247925a88ed3ffbb372e1b9448bfed"}
Jul 22 15:05:08.871001 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:08.870935    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d8g44" podStartSLOduration=9.509940918 podStartE2EDuration="46.870914546s" totalImagesPullingTime="37.360973628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:08.868353156 +0000 UTC m=+47.323066770" watchObservedRunningTime="2026-07-22 15:05:08.870914546 +0000 UTC m=+47.325628155"
Jul 22 15:05:09.855827 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:09.855798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:09.856211 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:09.855938    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 15:05:09.856211 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:09.856000    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls podName:df999d2c-1dd2-44d4-8731-63ec81feb9fb nodeName:}" failed. No retries permitted until 2026-07-22 15:05:13.85598514 +0000 UTC m=+52.310698733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-j5b5d" (UID: "df999d2c-1dd2-44d4-8731-63ec81feb9fb") : secret "samples-operator-tls" not found
Jul 22 15:05:10.159311 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:10.159275    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:10.159493 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:10.159351    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:10.159493 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:10.159460    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 15:05:10.159493 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:10.159484    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:14.159466345 +0000 UTC m=+52.614179938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : configmap references non-existent config key: service-ca.crt
Jul 22 15:05:10.159614 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:10.159510    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:14.159498635 +0000 UTC m=+52.614212228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : secret "router-metrics-certs-default" not found
Jul 22 15:05:10.851829 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:10.851749    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77" event={"ID":"880ce076-2799-46e2-b8d2-950e9f74dee0","Type":"ContainerStarted","Data":"de8f43a3e3ea4dff3abab4f7ab7fddbdb68067d642a64cc1af509f5b46bb2061"}
Jul 22 15:05:10.853492 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:10.853472    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/0.log"
Jul 22 15:05:10.853630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:10.853505    2560 generic.go:356] "Generic (PLEG): container finished" podID="d1269636-75f4-4267-b9d6-302fd715c279" containerID="899ed664626587b2d2eb681d28d5a7897d14c426774820ffe4951f3365aa9d9d" exitCode=255
Jul 22 15:05:10.853630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:10.853533    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv" event={"ID":"d1269636-75f4-4267-b9d6-302fd715c279","Type":"ContainerDied","Data":"899ed664626587b2d2eb681d28d5a7897d14c426774820ffe4951f3365aa9d9d"}
Jul 22 15:05:10.853806 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:10.853790    2560 scope.go:117] "RemoveContainer" containerID="899ed664626587b2d2eb681d28d5a7897d14c426774820ffe4951f3365aa9d9d"
Jul 22 15:05:10.871405 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:10.871365    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77" podStartSLOduration=1.385379537 podStartE2EDuration="4.871351237s" totalImagesPullingTime="3.4859717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:05:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:10.87079031 +0000 UTC m=+49.325503919" watchObservedRunningTime="2026-07-22 15:05:10.871351237 +0000 UTC m=+49.326064831"
Jul 22 15:05:11.857741 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:11.857713    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:05:11.858154 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:11.858139    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/0.log"
Jul 22 15:05:11.858203 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:11.858174    2560 generic.go:356] "Generic (PLEG): container finished" podID="d1269636-75f4-4267-b9d6-302fd715c279" containerID="78dcd922a9b8d68f451dabcf0f2d31a57fe674662a9bce0a39a30ee8996bcba9" exitCode=255
Jul 22 15:05:11.858263 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:11.858248    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv" event={"ID":"d1269636-75f4-4267-b9d6-302fd715c279","Type":"ContainerDied","Data":"78dcd922a9b8d68f451dabcf0f2d31a57fe674662a9bce0a39a30ee8996bcba9"}
Jul 22 15:05:11.858344 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:11.858333    2560 scope.go:117] "RemoveContainer" containerID="899ed664626587b2d2eb681d28d5a7897d14c426774820ffe4951f3365aa9d9d"
Jul 22 15:05:11.858571 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:11.858553    2560 scope.go:117] "RemoveContainer" containerID="78dcd922a9b8d68f451dabcf0f2d31a57fe674662a9bce0a39a30ee8996bcba9"
Jul 22 15:05:11.858776 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:11.858751    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-mnzqv_openshift-console-operator(d1269636-75f4-4267-b9d6-302fd715c279)\"" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv" podUID="d1269636-75f4-4267-b9d6-302fd715c279"
Jul 22 15:05:12.862281 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:12.862255    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:05:12.862651 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:12.862583    2560 scope.go:117] "RemoveContainer" containerID="78dcd922a9b8d68f451dabcf0f2d31a57fe674662a9bce0a39a30ee8996bcba9"
Jul 22 15:05:12.862745 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:12.862726    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-mnzqv_openshift-console-operator(d1269636-75f4-4267-b9d6-302fd715c279)\"" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv" podUID="d1269636-75f4-4267-b9d6-302fd715c279"
Jul 22 15:05:13.889977 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:13.889941    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:13.890339 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:13.890081    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 15:05:13.890339 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:13.890142    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls podName:df999d2c-1dd2-44d4-8731-63ec81feb9fb nodeName:}" failed. No retries permitted until 2026-07-22 15:05:21.890125961 +0000 UTC m=+60.344839558 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-j5b5d" (UID: "df999d2c-1dd2-44d4-8731-63ec81feb9fb") : secret "samples-operator-tls" not found
Jul 22 15:05:14.192339 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:14.192301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:14.192530 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:14.192352    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:05:14.192530 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:14.192405    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:14.192530 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:14.192499    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:22.192474471 +0000 UTC m=+60.647188066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : configmap references non-existent config key: service-ca.crt
Jul 22 15:05:14.192714 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:14.192547    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 15:05:14.192714 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:14.192562    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56df5f6fcb-jvxb8: secret "image-registry-tls" not found
Jul 22 15:05:14.192714 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:14.192607    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls podName:b9720254-d650-4e81-9417-3355eb50cce1 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:30.19259664 +0000 UTC m=+68.647310274 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls") pod "image-registry-56df5f6fcb-jvxb8" (UID: "b9720254-d650-4e81-9417-3355eb50cce1") : secret "image-registry-tls" not found
Jul 22 15:05:14.192714 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:14.192548    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 15:05:14.192714 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:14.192638    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:22.192629976 +0000 UTC m=+60.647343595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : secret "router-metrics-certs-default" not found
Jul 22 15:05:14.292841 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:14.292811    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:05:14.293011 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:14.292963    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 15:05:14.293078 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:14.293033    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls podName:08cc6e66-2c7b-483d-9d70-b96de5dd2c7a nodeName:}" failed. No retries permitted until 2026-07-22 15:05:30.293015044 +0000 UTC m=+68.747728637 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls") pod "dns-default-pblnf" (UID: "08cc6e66-2c7b-483d-9d70-b96de5dd2c7a") : secret "dns-default-metrics-tls" not found
Jul 22 15:05:16.708524 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:16.708481    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:16.709083 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:16.708550    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:16.709083 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:16.708966    2560 scope.go:117] "RemoveContainer" containerID="78dcd922a9b8d68f451dabcf0f2d31a57fe674662a9bce0a39a30ee8996bcba9"
Jul 22 15:05:16.709211 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:16.709165    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-mnzqv_openshift-console-operator(d1269636-75f4-4267-b9d6-302fd715c279)\"" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv" podUID="d1269636-75f4-4267-b9d6-302fd715c279"
Jul 22 15:05:17.437387 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:17.437359    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z4765_dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f/dns-node-resolver/0.log"
Jul 22 15:05:18.235676 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:18.235654    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hkpl4_3d1e6293-7310-4977-81ce-f804acdcbcd0/node-ca/0.log"
Jul 22 15:05:19.238511 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:19.238472    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gmpdw_6c583dee-6f25-4c83-9858-da1763f2a5f5/serve-healthcheck-canary/0.log"
Jul 22 15:05:21.955946 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:21.955900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:21.958118 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:21.958093    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/df999d2c-1dd2-44d4-8731-63ec81feb9fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-j5b5d\" (UID: \"df999d2c-1dd2-44d4-8731-63ec81feb9fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:22.258005 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:22.257925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:22.258005 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:22.257989    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:22.258228 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:22.258084    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 15:05:22.258228 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:22.258099    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:38.258078006 +0000 UTC m=+76.712791618 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : configmap references non-existent config key: service-ca.crt
Jul 22 15:05:22.258228 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:22.258130    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs podName:9e0637ad-06fc-496c-8a90-884a39cf26d9 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:38.258119728 +0000 UTC m=+76.712833335 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs") pod "router-default-d789dc46-blhfw" (UID: "9e0637ad-06fc-496c-8a90-884a39cf26d9") : secret "router-metrics-certs-default" not found
Jul 22 15:05:22.258228 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:22.258129    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rs8z7\""
Jul 22 15:05:22.265530 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:22.265512    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"
Jul 22 15:05:22.384158 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:22.384137    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d"]
Jul 22 15:05:22.887905 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:22.887861    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d" event={"ID":"df999d2c-1dd2-44d4-8731-63ec81feb9fb","Type":"ContainerStarted","Data":"d2453671eca38b3dc6db216065a715f301be7660b5f99da9fd2c030891db010b"}
Jul 22 15:05:24.805618 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:24.805593    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf658"
Jul 22 15:05:24.896216 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:24.896182    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d" event={"ID":"df999d2c-1dd2-44d4-8731-63ec81feb9fb","Type":"ContainerStarted","Data":"e84b32b244cfd25b6d678d14418b998188939827a5a338a014616e10d53d5f58"}
Jul 22 15:05:24.896400 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:24.896257    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d" event={"ID":"df999d2c-1dd2-44d4-8731-63ec81feb9fb","Type":"ContainerStarted","Data":"29dc09e4a2e44d92539158b873ba284f823854bc0516636aad38fade045b43e0"}
Jul 22 15:05:24.924414 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:24.924365    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-j5b5d" podStartSLOduration=17.204528291 podStartE2EDuration="18.924350876s" totalImagesPullingTime="1.719822585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:05:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:24.924307217 +0000 UTC m=+63.379020842" watchObservedRunningTime="2026-07-22 15:05:24.924350876 +0000 UTC m=+63.379064469"
Jul 22 15:05:27.803419 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.803376    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:05:27.803882 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.803431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:05:27.806967 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.806950    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 15:05:27.808001 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.807987    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 15:05:27.816278 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.816255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba27779c-af36-4803-88e3-0f0f25a01210-metrics-certs\") pod \"network-metrics-daemon-w9tkp\" (UID: \"ba27779c-af36-4803-88e3-0f0f25a01210\") " pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:05:27.816393 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.816259    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b0d8426-5e95-49b9-b057-b5e5cfc59182-original-pull-secret\") pod \"global-pull-secret-syncer-c8jkn\" (UID: \"2b0d8426-5e95-49b9-b057-b5e5cfc59182\") " pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:05:27.826994 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.826974    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zbjvd\""
Jul 22 15:05:27.835255 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.835239    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9tkp"
Jul 22 15:05:27.904136 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.904101    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:05:27.906866 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.906844    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 15:05:27.924341 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.924319    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 15:05:27.927939 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.927915    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnh2n\" (UniqueName: \"kubernetes.io/projected/35ef3246-ba7b-44fd-8a27-d59603445a9a-kube-api-access-dnh2n\") pod \"network-check-target-8ckgj\" (UID: \"35ef3246-ba7b-44fd-8a27-d59603445a9a\") " pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:05:27.957320 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:27.957296    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w9tkp"]
Jul 22 15:05:27.960045 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:27.960020    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba27779c_af36_4803_88e3_0f0f25a01210.slice/crio-7cd0a9258b3b8a52cdc561b281473b49bcf8317c3cca81f4f3f189eb14f22e66 WatchSource:0}: Error finding container 7cd0a9258b3b8a52cdc561b281473b49bcf8317c3cca81f4f3f189eb14f22e66: Status 404 returned error can't find the container with id 7cd0a9258b3b8a52cdc561b281473b49bcf8317c3cca81f4f3f189eb14f22e66
Jul 22 15:05:28.108797 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:28.108737    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8jkn"
Jul 22 15:05:28.119894 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:28.119875    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9ldrl\""
Jul 22 15:05:28.127186 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:28.127168    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:05:28.238650 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:28.238620    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c8jkn"]
Jul 22 15:05:28.241663 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:28.241638    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b0d8426_5e95_49b9_b057_b5e5cfc59182.slice/crio-94e1c2272c21e53ab39e52aa69040eb46b034a12080ace496fc0937fcef5780f WatchSource:0}: Error finding container 94e1c2272c21e53ab39e52aa69040eb46b034a12080ace496fc0937fcef5780f: Status 404 returned error can't find the container with id 94e1c2272c21e53ab39e52aa69040eb46b034a12080ace496fc0937fcef5780f
Jul 22 15:05:28.257634 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:28.257614    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ckgj"]
Jul 22 15:05:28.260035 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:28.260012    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ef3246_ba7b_44fd_8a27_d59603445a9a.slice/crio-cefe37a1629e3b7a8b53f754458c8f0fab4c51dc1142e484b6e3f04095d7e3eb WatchSource:0}: Error finding container cefe37a1629e3b7a8b53f754458c8f0fab4c51dc1142e484b6e3f04095d7e3eb: Status 404 returned error can't find the container with id cefe37a1629e3b7a8b53f754458c8f0fab4c51dc1142e484b6e3f04095d7e3eb
Jul 22 15:05:28.911652 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:28.911608    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w9tkp" event={"ID":"ba27779c-af36-4803-88e3-0f0f25a01210","Type":"ContainerStarted","Data":"7c6aec64725185f84105718ec2e507224ebde03efd77c7379b3505d08da4ef88"}
Jul 22 15:05:28.912098 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:28.911684    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w9tkp" event={"ID":"ba27779c-af36-4803-88e3-0f0f25a01210","Type":"ContainerStarted","Data":"7cd0a9258b3b8a52cdc561b281473b49bcf8317c3cca81f4f3f189eb14f22e66"}
Jul 22 15:05:28.913019 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:28.912979    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c8jkn" event={"ID":"2b0d8426-5e95-49b9-b057-b5e5cfc59182","Type":"ContainerStarted","Data":"94e1c2272c21e53ab39e52aa69040eb46b034a12080ace496fc0937fcef5780f"}
Jul 22 15:05:28.914552 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:28.914525    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ckgj" event={"ID":"35ef3246-ba7b-44fd-8a27-d59603445a9a","Type":"ContainerStarted","Data":"cefe37a1629e3b7a8b53f754458c8f0fab4c51dc1142e484b6e3f04095d7e3eb"}
Jul 22 15:05:29.920084 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:29.920048    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w9tkp" event={"ID":"ba27779c-af36-4803-88e3-0f0f25a01210","Type":"ContainerStarted","Data":"44f580027e5f3f9d638269975d3de3a7a94397f645f22523911494c3dcdb349e"}
Jul 22 15:05:29.943718 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:29.943663    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w9tkp" podStartSLOduration=67.112820075 podStartE2EDuration="1m7.943642621s" totalImagesPullingTime="830.822546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:29.943186914 +0000 UTC m=+68.397900529" watchObservedRunningTime="2026-07-22 15:05:29.943642621 +0000 UTC m=+68.398356235"
Jul 22 15:05:30.225252 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:30.225156    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:05:30.228139 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:30.228108    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"image-registry-56df5f6fcb-jvxb8\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") " pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:05:30.326244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:30.326207    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:05:30.328691 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:30.328669    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08cc6e66-2c7b-483d-9d70-b96de5dd2c7a-metrics-tls\") pod \"dns-default-pblnf\" (UID: \"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a\") " pod="openshift-dns/dns-default-pblnf"
Jul 22 15:05:30.493648 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:30.493566    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xgdsc\""
Jul 22 15:05:30.501778 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:30.501710    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:05:30.530804 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:30.530778    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s4xt5\""
Jul 22 15:05:30.538323 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:30.538302    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pblnf"
Jul 22 15:05:31.445775 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.445584    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pblnf"]
Jul 22 15:05:31.451054 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:31.451024    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08cc6e66_2c7b_483d_9d70_b96de5dd2c7a.slice/crio-c0f0dc1691fb58ca8cdb2dddf2d39dc3805ae76bf4ce7d4402f2ff61776a4be7 WatchSource:0}: Error finding container c0f0dc1691fb58ca8cdb2dddf2d39dc3805ae76bf4ce7d4402f2ff61776a4be7: Status 404 returned error can't find the container with id c0f0dc1691fb58ca8cdb2dddf2d39dc3805ae76bf4ce7d4402f2ff61776a4be7
Jul 22 15:05:31.466503 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.466416    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56df5f6fcb-jvxb8"]
Jul 22 15:05:31.530384 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:31.530352    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9720254_d650_4e81_9417_3355eb50cce1.slice/crio-cacf7f814108216fb69395d5d3c235948bd2983ad56452ff001d132f65a677ea WatchSource:0}: Error finding container cacf7f814108216fb69395d5d3c235948bd2983ad56452ff001d132f65a677ea: Status 404 returned error can't find the container with id cacf7f814108216fb69395d5d3c235948bd2983ad56452ff001d132f65a677ea
Jul 22 15:05:31.596724 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.596705    2560 scope.go:117] "RemoveContainer" containerID="78dcd922a9b8d68f451dabcf0f2d31a57fe674662a9bce0a39a30ee8996bcba9"
Jul 22 15:05:31.928898 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.928868    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:05:31.929069 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.928959    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv" event={"ID":"d1269636-75f4-4267-b9d6-302fd715c279","Type":"ContainerStarted","Data":"6eec87a41123110df3a458bcd5c38a327a630a4387629a2259446810dac28ee8"}
Jul 22 15:05:31.929464 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.929430    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:31.931095 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.931069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ckgj" event={"ID":"35ef3246-ba7b-44fd-8a27-d59603445a9a","Type":"ContainerStarted","Data":"efab3a0ec39b12e2d8719c74e75633843e907ae1b88decc55dc6b68db8df933e"}
Jul 22 15:05:31.931224 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.931204    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:05:31.932887 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.932856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8" event={"ID":"b9720254-d650-4e81-9417-3355eb50cce1","Type":"ContainerStarted","Data":"d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f"}
Jul 22 15:05:31.933004 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.932895    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8" event={"ID":"b9720254-d650-4e81-9417-3355eb50cce1","Type":"ContainerStarted","Data":"cacf7f814108216fb69395d5d3c235948bd2983ad56452ff001d132f65a677ea"}
Jul 22 15:05:31.933234 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.933213    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:05:31.934231 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.934206    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pblnf" event={"ID":"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a","Type":"ContainerStarted","Data":"c0f0dc1691fb58ca8cdb2dddf2d39dc3805ae76bf4ce7d4402f2ff61776a4be7"}
Jul 22 15:05:31.951701 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.951429    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv" podStartSLOduration=22.57936436 podStartE2EDuration="25.951409797s" totalImagesPullingTime="3.372045437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:05:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:31.949577953 +0000 UTC m=+70.404291596" watchObservedRunningTime="2026-07-22 15:05:31.951409797 +0000 UTC m=+70.406123417"
Jul 22 15:05:31.972968 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.972635    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8" podStartSLOduration=44.972614339 podStartE2EDuration="44.972614339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:31.970810579 +0000 UTC m=+70.425524198" watchObservedRunningTime="2026-07-22 15:05:31.972614339 +0000 UTC m=+70.427327955"
Jul 22 15:05:31.986968 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:31.986906    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8ckgj" podStartSLOduration=66.702257405 podStartE2EDuration="1m9.986885381s" totalImagesPullingTime="3.284627976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:31.986756109 +0000 UTC m=+70.441469752" watchObservedRunningTime="2026-07-22 15:05:31.986885381 +0000 UTC m=+70.441598998"
Jul 22 15:05:32.715643 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:32.715596    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-mnzqv"
Jul 22 15:05:34.946660 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:34.946617    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pblnf" event={"ID":"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a","Type":"ContainerStarted","Data":"3568979b9b6cd30b91971f30e29158568deabfc389581af457d669aedbe063b1"}
Jul 22 15:05:34.947113 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:34.946672    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pblnf" event={"ID":"08cc6e66-2c7b-483d-9d70-b96de5dd2c7a","Type":"ContainerStarted","Data":"13f5fbcb775d3cc32ed2de13e2e11fbeb5b891cf6d5208ce70f12079342cfa55"}
Jul 22 15:05:34.947113 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:34.946710    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pblnf"
Jul 22 15:05:34.948101 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:34.948078    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c8jkn" event={"ID":"2b0d8426-5e95-49b9-b057-b5e5cfc59182","Type":"ContainerStarted","Data":"8eea62d6596aaa7c317dab3a2499f07e211d11df7e21d5579540cad88d4afa2d"}
Jul 22 15:05:34.980482 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:34.980418    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pblnf" podStartSLOduration=34.554306117 podStartE2EDuration="36.980406687s" totalImagesPullingTime="2.42610057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:04:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:34.965177641 +0000 UTC m=+73.419891257" watchObservedRunningTime="2026-07-22 15:05:34.980406687 +0000 UTC m=+73.435120329"
Jul 22 15:05:34.980696 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:34.980675    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c8jkn" podStartSLOduration=67.340198693 podStartE2EDuration="1m12.980670696s" totalImagesPullingTime="5.640472003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:34.979927771 +0000 UTC m=+73.434641401" watchObservedRunningTime="2026-07-22 15:05:34.980670696 +0000 UTC m=+73.435384368"
Jul 22 15:05:36.022995 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.022963    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"]
Jul 22 15:05:36.025646 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.025626    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.028570 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.028546    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 22 15:05:36.028570 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.028563    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 22 15:05:36.029769 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.029751    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 22 15:05:36.029845 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.029755    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 22 15:05:36.047403 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.047378    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"]
Jul 22 15:05:36.067994 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.067962    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b505ffc1-ea8c-48a7-9eda-d94cd87358ef-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc96b848b-7vmf4\" (UID: \"b505ffc1-ea8c-48a7-9eda-d94cd87358ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.068150 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.068015    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b505ffc1-ea8c-48a7-9eda-d94cd87358ef-tmp\") pod \"klusterlet-addon-workmgr-5fc96b848b-7vmf4\" (UID: \"b505ffc1-ea8c-48a7-9eda-d94cd87358ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.068201 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.068165    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n28x\" (UniqueName: \"kubernetes.io/projected/b505ffc1-ea8c-48a7-9eda-d94cd87358ef-kube-api-access-2n28x\") pod \"klusterlet-addon-workmgr-5fc96b848b-7vmf4\" (UID: \"b505ffc1-ea8c-48a7-9eda-d94cd87358ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.108029 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.107991    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-pxqhr"]
Jul 22 15:05:36.110849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.110828    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-pxqhr"
Jul 22 15:05:36.118751 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.118724    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 22 15:05:36.118872 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.118825    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-shhtt\""
Jul 22 15:05:36.118872 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.118738    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 22 15:05:36.120046 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.120027    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pbrrl"]
Jul 22 15:05:36.122800 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.122786    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.127237 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.127216    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 15:05:36.127350 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.127286    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 15:05:36.127350 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.127263    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t78n4\""
Jul 22 15:05:36.127479 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.127266    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 15:05:36.127689 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.127670    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 15:05:36.129895 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.129861    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-pxqhr"]
Jul 22 15:05:36.142418 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.142392    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pbrrl"]
Jul 22 15:05:36.169171 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b713f6e2-248e-4563-b2c3-682c1fc41469-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.169313 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b505ffc1-ea8c-48a7-9eda-d94cd87358ef-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc96b848b-7vmf4\" (UID: \"b505ffc1-ea8c-48a7-9eda-d94cd87358ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.169313 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169203    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b505ffc1-ea8c-48a7-9eda-d94cd87358ef-tmp\") pod \"klusterlet-addon-workmgr-5fc96b848b-7vmf4\" (UID: \"b505ffc1-ea8c-48a7-9eda-d94cd87358ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.169313 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169221    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b713f6e2-248e-4563-b2c3-682c1fc41469-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.169313 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b713f6e2-248e-4563-b2c3-682c1fc41469-data-volume\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.169556 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169363    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b713f6e2-248e-4563-b2c3-682c1fc41469-crio-socket\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.169556 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169406    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgrqk\" (UniqueName: \"kubernetes.io/projected/41fe00d2-2f54-4e8f-b445-91a5e4da5936-kube-api-access-zgrqk\") pod \"downloads-5bc67dcd46-pxqhr\" (UID: \"41fe00d2-2f54-4e8f-b445-91a5e4da5936\") " pod="openshift-console/downloads-5bc67dcd46-pxqhr"
Jul 22 15:05:36.169556 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169463    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jbn9\" (UniqueName: \"kubernetes.io/projected/b713f6e2-248e-4563-b2c3-682c1fc41469-kube-api-access-4jbn9\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.169556 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169502    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n28x\" (UniqueName: \"kubernetes.io/projected/b505ffc1-ea8c-48a7-9eda-d94cd87358ef-kube-api-access-2n28x\") pod \"klusterlet-addon-workmgr-5fc96b848b-7vmf4\" (UID: \"b505ffc1-ea8c-48a7-9eda-d94cd87358ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.169798 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.169631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b505ffc1-ea8c-48a7-9eda-d94cd87358ef-tmp\") pod \"klusterlet-addon-workmgr-5fc96b848b-7vmf4\" (UID: \"b505ffc1-ea8c-48a7-9eda-d94cd87358ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.171673 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.171658    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b505ffc1-ea8c-48a7-9eda-d94cd87358ef-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc96b848b-7vmf4\" (UID: \"b505ffc1-ea8c-48a7-9eda-d94cd87358ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.244255 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.244220    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n28x\" (UniqueName: \"kubernetes.io/projected/b505ffc1-ea8c-48a7-9eda-d94cd87358ef-kube-api-access-2n28x\") pod \"klusterlet-addon-workmgr-5fc96b848b-7vmf4\" (UID: \"b505ffc1-ea8c-48a7-9eda-d94cd87358ef\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.270225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.270196    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b713f6e2-248e-4563-b2c3-682c1fc41469-crio-socket\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.270225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.270227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgrqk\" (UniqueName: \"kubernetes.io/projected/41fe00d2-2f54-4e8f-b445-91a5e4da5936-kube-api-access-zgrqk\") pod \"downloads-5bc67dcd46-pxqhr\" (UID: \"41fe00d2-2f54-4e8f-b445-91a5e4da5936\") " pod="openshift-console/downloads-5bc67dcd46-pxqhr"
Jul 22 15:05:36.270382 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.270248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jbn9\" (UniqueName: \"kubernetes.io/projected/b713f6e2-248e-4563-b2c3-682c1fc41469-kube-api-access-4jbn9\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.270382 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.270280    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b713f6e2-248e-4563-b2c3-682c1fc41469-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.270382 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.270304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b713f6e2-248e-4563-b2c3-682c1fc41469-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.270382 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.270313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b713f6e2-248e-4563-b2c3-682c1fc41469-crio-socket\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.270382 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.270330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b713f6e2-248e-4563-b2c3-682c1fc41469-data-volume\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.270670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.270652    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b713f6e2-248e-4563-b2c3-682c1fc41469-data-volume\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.270903 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.270884    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b713f6e2-248e-4563-b2c3-682c1fc41469-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.272522 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.272505    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b713f6e2-248e-4563-b2c3-682c1fc41469-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.301024 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.300965    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgrqk\" (UniqueName: \"kubernetes.io/projected/41fe00d2-2f54-4e8f-b445-91a5e4da5936-kube-api-access-zgrqk\") pod \"downloads-5bc67dcd46-pxqhr\" (UID: \"41fe00d2-2f54-4e8f-b445-91a5e4da5936\") " pod="openshift-console/downloads-5bc67dcd46-pxqhr"
Jul 22 15:05:36.306858 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.306837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jbn9\" (UniqueName: \"kubernetes.io/projected/b713f6e2-248e-4563-b2c3-682c1fc41469-kube-api-access-4jbn9\") pod \"insights-runtime-extractor-pbrrl\" (UID: \"b713f6e2-248e-4563-b2c3-682c1fc41469\") " pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.334855 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.334832    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:36.419978 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.419714    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-pxqhr"
Jul 22 15:05:36.435696 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.435669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pbrrl"
Jul 22 15:05:36.471382 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.471173    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"]
Jul 22 15:05:36.476660 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:36.476456    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb505ffc1_ea8c_48a7_9eda_d94cd87358ef.slice/crio-ba22c3aa869342ade8ffc65837e243487676b56f12339917e78f07caf810ead2 WatchSource:0}: Error finding container ba22c3aa869342ade8ffc65837e243487676b56f12339917e78f07caf810ead2: Status 404 returned error can't find the container with id ba22c3aa869342ade8ffc65837e243487676b56f12339917e78f07caf810ead2
Jul 22 15:05:36.584656 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.584570    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-pxqhr"]
Jul 22 15:05:36.588285 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:36.588255    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41fe00d2_2f54_4e8f_b445_91a5e4da5936.slice/crio-d3e3e5979ce592e8f6461803acb4ee56dc3a1519edfc09a9d5ec27f9c7c7139e WatchSource:0}: Error finding container d3e3e5979ce592e8f6461803acb4ee56dc3a1519edfc09a9d5ec27f9c7c7139e: Status 404 returned error can't find the container with id d3e3e5979ce592e8f6461803acb4ee56dc3a1519edfc09a9d5ec27f9c7c7139e
Jul 22 15:05:36.610537 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.610513    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pbrrl"]
Jul 22 15:05:36.614954 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:36.614931    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb713f6e2_248e_4563_b2c3_682c1fc41469.slice/crio-f49e6f95a322e09e309ab4b3abf8bd9b73e95135218c00a72b8725208a704a8e WatchSource:0}: Error finding container f49e6f95a322e09e309ab4b3abf8bd9b73e95135218c00a72b8725208a704a8e: Status 404 returned error can't find the container with id f49e6f95a322e09e309ab4b3abf8bd9b73e95135218c00a72b8725208a704a8e
Jul 22 15:05:36.955806 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.955772    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbrrl" event={"ID":"b713f6e2-248e-4563-b2c3-682c1fc41469","Type":"ContainerStarted","Data":"f49e6f95a322e09e309ab4b3abf8bd9b73e95135218c00a72b8725208a704a8e"}
Jul 22 15:05:36.956897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.956871    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4" event={"ID":"b505ffc1-ea8c-48a7-9eda-d94cd87358ef","Type":"ContainerStarted","Data":"ba22c3aa869342ade8ffc65837e243487676b56f12339917e78f07caf810ead2"}
Jul 22 15:05:36.957899 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:36.957867    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-pxqhr" event={"ID":"41fe00d2-2f54-4e8f-b445-91a5e4da5936","Type":"ContainerStarted","Data":"d3e3e5979ce592e8f6461803acb4ee56dc3a1519edfc09a9d5ec27f9c7c7139e"}
Jul 22 15:05:38.288646 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:38.288570    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:38.288646 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:38.288650    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:38.289416 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:38.289388    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e0637ad-06fc-496c-8a90-884a39cf26d9-service-ca-bundle\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:38.291637 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:38.291614    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0637ad-06fc-496c-8a90-884a39cf26d9-metrics-certs\") pod \"router-default-d789dc46-blhfw\" (UID: \"9e0637ad-06fc-496c-8a90-884a39cf26d9\") " pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:38.491391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:38.491355    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g4j4d\""
Jul 22 15:05:38.499694 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:38.499498    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:39.226949 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:39.226917    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-d789dc46-blhfw"]
Jul 22 15:05:39.230334 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:39.230308    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e0637ad_06fc_496c_8a90_884a39cf26d9.slice/crio-e0c47ed94dab35f1610b02e1e223893aa255b1f077c1270ae1cff1d0efbfad01 WatchSource:0}: Error finding container e0c47ed94dab35f1610b02e1e223893aa255b1f077c1270ae1cff1d0efbfad01: Status 404 returned error can't find the container with id e0c47ed94dab35f1610b02e1e223893aa255b1f077c1270ae1cff1d0efbfad01
Jul 22 15:05:39.975352 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:39.975301    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-d789dc46-blhfw" event={"ID":"9e0637ad-06fc-496c-8a90-884a39cf26d9","Type":"ContainerStarted","Data":"fef0fed4455584f6701f804f05ffbc462971fbef583d9b0d1a860e3fdf0735b0"}
Jul 22 15:05:39.975849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:39.975407    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-d789dc46-blhfw" event={"ID":"9e0637ad-06fc-496c-8a90-884a39cf26d9","Type":"ContainerStarted","Data":"e0c47ed94dab35f1610b02e1e223893aa255b1f077c1270ae1cff1d0efbfad01"}
Jul 22 15:05:40.006604 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:40.006524    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-d789dc46-blhfw" podStartSLOduration=34.006504441 podStartE2EDuration="34.006504441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:05:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:40.004218905 +0000 UTC m=+78.458932542" watchObservedRunningTime="2026-07-22 15:05:40.006504441 +0000 UTC m=+78.461218057"
Jul 22 15:05:40.499936 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:40.499869    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:40.502962 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:40.502869    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:40.985422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:40.985348    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4" event={"ID":"b505ffc1-ea8c-48a7-9eda-d94cd87358ef","Type":"ContainerStarted","Data":"6997006b27787fbef698a48f60a99bd1976b8847c60567bddf3a88492560d429"}
Jul 22 15:05:40.987082 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:40.986694    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:40.987082 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:40.986937    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:40.987728 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:40.987590    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-d789dc46-blhfw"
Jul 22 15:05:40.988270 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:40.988230    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4"
Jul 22 15:05:41.029295 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:41.028740    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc96b848b-7vmf4" podStartSLOduration=1.724453972 podStartE2EDuration="6.028719282s" totalImagesPullingTime="4.30426531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:41.028562761 +0000 UTC m=+79.483276380" watchObservedRunningTime="2026-07-22 15:05:41.028719282 +0000 UTC m=+79.483432899"
Jul 22 15:05:44.953910 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:44.953353    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pblnf"
Jul 22 15:05:47.011488 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:47.011434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbrrl" event={"ID":"b713f6e2-248e-4563-b2c3-682c1fc41469","Type":"ContainerStarted","Data":"f5ed1e1b5ed84bb35e38cfd29feabf92ef2be336f0ca8dab06f06ab426a0c54d"}
Jul 22 15:05:53.946686 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:53.946647    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:05:55.047181 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:55.046157    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-pxqhr" event={"ID":"41fe00d2-2f54-4e8f-b445-91a5e4da5936","Type":"ContainerStarted","Data":"6348d2171a8059a2faf50be2397d8155ef3e5772d6ed2023b83d4cb3189fce5f"}
Jul 22 15:05:55.047181 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:55.047132    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-pxqhr"
Jul 22 15:05:55.049056 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:55.048701    2560 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-pxqhr container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.14:8080/\": dial tcp 10.132.0.14:8080: connect: connection refused" start-of-body=
Jul 22 15:05:55.049056 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:55.048761    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-pxqhr" podUID="41fe00d2-2f54-4e8f-b445-91a5e4da5936" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.14:8080/\": dial tcp 10.132.0.14:8080: connect: connection refused"
Jul 22 15:05:55.069580 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:55.069209    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-pxqhr" podStartSLOduration=0.815542519 podStartE2EDuration="19.069174571s" totalImagesPullingTime="18.253632052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:05:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:05:55.067981364 +0000 UTC m=+93.522694981" watchObservedRunningTime="2026-07-22 15:05:55.069174571 +0000 UTC m=+93.523888187"
Jul 22 15:05:56.066541 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:56.066503    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-pxqhr"
Jul 22 15:05:58.998053 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:58.998018    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tvwsg"]
Jul 22 15:05:59.019306 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.019279    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.022888 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.022857    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 15:05:59.023024 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.022907    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 15:05:59.023024 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.022949    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 15:05:59.023130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.023024    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 15:05:59.024187 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.024149    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mxpdq\""
Jul 22 15:05:59.024304 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.024204    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 15:05:59.024304 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.024257    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 15:05:59.162551 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.162516    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/251ab756-226a-491c-ab81-d021165a4738-node-exporter-accelerators-collector-config\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.162729 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.162577    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/251ab756-226a-491c-ab81-d021165a4738-node-exporter-textfile\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.162729 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.162663    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/251ab756-226a-491c-ab81-d021165a4738-node-exporter-tls\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.162729 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.162710    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/251ab756-226a-491c-ab81-d021165a4738-root\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.162884 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.162738    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72j95\" (UniqueName: \"kubernetes.io/projected/251ab756-226a-491c-ab81-d021165a4738-kube-api-access-72j95\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.162884 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.162795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/251ab756-226a-491c-ab81-d021165a4738-node-exporter-wtmp\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.162884 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.162836    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/251ab756-226a-491c-ab81-d021165a4738-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.162884 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.162862    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/251ab756-226a-491c-ab81-d021165a4738-sys\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.163039 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.162915    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/251ab756-226a-491c-ab81-d021165a4738-metrics-client-ca\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.177547 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.177514    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56df5f6fcb-jvxb8"]
Jul 22 15:05:59.263555 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263475    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/251ab756-226a-491c-ab81-d021165a4738-node-exporter-accelerators-collector-config\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.263555 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263541    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/251ab756-226a-491c-ab81-d021165a4738-node-exporter-textfile\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.263782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263578    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/251ab756-226a-491c-ab81-d021165a4738-node-exporter-tls\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.263782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/251ab756-226a-491c-ab81-d021165a4738-root\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.263782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263635    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72j95\" (UniqueName: \"kubernetes.io/projected/251ab756-226a-491c-ab81-d021165a4738-kube-api-access-72j95\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.263782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263666    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/251ab756-226a-491c-ab81-d021165a4738-node-exporter-wtmp\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.263782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263698    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/251ab756-226a-491c-ab81-d021165a4738-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.263782 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:59.263715    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 22 15:05:59.263782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263726    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/251ab756-226a-491c-ab81-d021165a4738-sys\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.263782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263735    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/251ab756-226a-491c-ab81-d021165a4738-root\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.263782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263763    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/251ab756-226a-491c-ab81-d021165a4738-metrics-client-ca\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.264234 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:05:59.263792    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/251ab756-226a-491c-ab81-d021165a4738-node-exporter-tls podName:251ab756-226a-491c-ab81-d021165a4738 nodeName:}" failed. No retries permitted until 2026-07-22 15:05:59.763772366 +0000 UTC m=+98.218485963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/251ab756-226a-491c-ab81-d021165a4738-node-exporter-tls") pod "node-exporter-tvwsg" (UID: "251ab756-226a-491c-ab81-d021165a4738") : secret "node-exporter-tls" not found
Jul 22 15:05:59.264234 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263904    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/251ab756-226a-491c-ab81-d021165a4738-node-exporter-wtmp\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.264234 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.263938    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/251ab756-226a-491c-ab81-d021165a4738-sys\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.264234 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.264132    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/251ab756-226a-491c-ab81-d021165a4738-node-exporter-accelerators-collector-config\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.264429 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.264269    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/251ab756-226a-491c-ab81-d021165a4738-node-exporter-textfile\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.264619 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.264599    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/251ab756-226a-491c-ab81-d021165a4738-metrics-client-ca\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.266464 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.266429    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/251ab756-226a-491c-ab81-d021165a4738-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.284908 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.284812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72j95\" (UniqueName: \"kubernetes.io/projected/251ab756-226a-491c-ab81-d021165a4738-kube-api-access-72j95\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.768469 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.768411    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/251ab756-226a-491c-ab81-d021165a4738-node-exporter-tls\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.771038 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.771008    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/251ab756-226a-491c-ab81-d021165a4738-node-exporter-tls\") pod \"node-exporter-tvwsg\" (UID: \"251ab756-226a-491c-ab81-d021165a4738\") " pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.930339 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:05:59.930298    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tvwsg"
Jul 22 15:05:59.978818 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:05:59.978787    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod251ab756_226a_491c_ab81_d021165a4738.slice/crio-8e0ac81f887bcb7c0aedce1a31d2c6f3b8ae6f7d4a69970c36f95713afa816a7 WatchSource:0}: Error finding container 8e0ac81f887bcb7c0aedce1a31d2c6f3b8ae6f7d4a69970c36f95713afa816a7: Status 404 returned error can't find the container with id 8e0ac81f887bcb7c0aedce1a31d2c6f3b8ae6f7d4a69970c36f95713afa816a7
Jul 22 15:06:00.071944 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.071809    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbrrl" event={"ID":"b713f6e2-248e-4563-b2c3-682c1fc41469","Type":"ContainerStarted","Data":"ed1cec9ff75fc06756a2e4e99a818f7725ab965f91b671f2d0867d5d16f38afd"}
Jul 22 15:06:00.073879 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.073815    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tvwsg" event={"ID":"251ab756-226a-491c-ab81-d021165a4738","Type":"ContainerStarted","Data":"8e0ac81f887bcb7c0aedce1a31d2c6f3b8ae6f7d4a69970c36f95713afa816a7"}
Jul 22 15:06:00.150223 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.149535    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:06:00.174464 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.174421    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.179299 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.178624    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 22 15:06:00.179299 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.178847    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 22 15:06:00.179299 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.178999    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 22 15:06:00.179299 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.179160    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 22 15:06:00.181287 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.180765    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 22 15:06:00.181287 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.181110    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j6nw7\""
Jul 22 15:06:00.181287 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.181125    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:06:00.181543 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.181466    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 22 15:06:00.182835 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.181792    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 22 15:06:00.182835 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.182076    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 22 15:06:00.182835 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.182268    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 22 15:06:00.270717 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.270717 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270723    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.270956 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270753    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k8qz\" (UniqueName: \"kubernetes.io/projected/7096c167-bdc7-408b-acea-15439fe7b559-kube-api-access-4k8qz\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.270956 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270771    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7096c167-bdc7-408b-acea-15439fe7b559-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.270956 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270790    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.270956 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270812    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7096c167-bdc7-408b-acea-15439fe7b559-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.270956 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270837    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-config-volume\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.270956 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7096c167-bdc7-408b-acea-15439fe7b559-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.270956 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270931    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.271298 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270961    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-web-config\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.271298 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.270983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7096c167-bdc7-408b-acea-15439fe7b559-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.271298 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.271016    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7096c167-bdc7-408b-acea-15439fe7b559-config-out\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.271298 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.271039    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.371997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.371872    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.371997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.371928    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-web-config\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.371997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.371956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7096c167-bdc7-408b-acea-15439fe7b559-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.371997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.371986    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7096c167-bdc7-408b-acea-15439fe7b559-config-out\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.372309 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372004    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.372309 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.372309 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372071    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.372309 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372101    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k8qz\" (UniqueName: \"kubernetes.io/projected/7096c167-bdc7-408b-acea-15439fe7b559-kube-api-access-4k8qz\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.372309 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372130    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7096c167-bdc7-408b-acea-15439fe7b559-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.372309 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372155    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.372309 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7096c167-bdc7-408b-acea-15439fe7b559-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.372309 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372205    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-config-volume\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.372309 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372246    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7096c167-bdc7-408b-acea-15439fe7b559-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.373042 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.372989    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7096c167-bdc7-408b-acea-15439fe7b559-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.376997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.373666    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7096c167-bdc7-408b-acea-15439fe7b559-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.376997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.374236    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7096c167-bdc7-408b-acea-15439fe7b559-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.376997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.376314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-web-config\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.376997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.376727    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.377271 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.377067    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.377579 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.377535    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7096c167-bdc7-408b-acea-15439fe7b559-config-out\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.377691 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.377658    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.377807 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.377770    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.381109 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.381038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7096c167-bdc7-408b-acea-15439fe7b559-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.381234 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.381161    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.382254 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.382232    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7096c167-bdc7-408b-acea-15439fe7b559-config-volume\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.385503 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.385438    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k8qz\" (UniqueName: \"kubernetes.io/projected/7096c167-bdc7-408b-acea-15439fe7b559-kube-api-access-4k8qz\") pod \"alertmanager-main-0\" (UID: \"7096c167-bdc7-408b-acea-15439fe7b559\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.495903 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.495870    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 15:06:00.708223 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:00.707161    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 15:06:00.710679 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:06:00.710460    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7096c167_bdc7_408b_acea_15439fe7b559.slice/crio-66df9770fa6d57969d32adf8df15e4be92cc7b751af9cb67abb67ef0cbd21567 WatchSource:0}: Error finding container 66df9770fa6d57969d32adf8df15e4be92cc7b751af9cb67abb67ef0cbd21567: Status 404 returned error can't find the container with id 66df9770fa6d57969d32adf8df15e4be92cc7b751af9cb67abb67ef0cbd21567
Jul 22 15:06:01.008991 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.006392    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-db8557d74-dcchm"]
Jul 22 15:06:01.032860 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.032829    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-db8557d74-dcchm"]
Jul 22 15:06:01.033085 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.033059    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.038293 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.036838    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 22 15:06:01.038293 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.037002    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5pcbl2rppgs9h\""
Jul 22 15:06:01.038293 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.037128    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 22 15:06:01.038293 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.037205    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 22 15:06:01.038293 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.037298    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 22 15:06:01.038293 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.037369    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xg7nv\""
Jul 22 15:06:01.038293 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.037483    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 22 15:06:01.081890 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.081856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7096c167-bdc7-408b-acea-15439fe7b559","Type":"ContainerStarted","Data":"66df9770fa6d57969d32adf8df15e4be92cc7b751af9cb67abb67ef0cbd21567"}
Jul 22 15:06:01.181684 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.181565    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.182065 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.181691    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-grpc-tls\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.182065 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.181806    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.182065 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.181872    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-metrics-client-ca\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.182065 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.182005    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.182065 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.182040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.182336 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.182086    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-tls\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.182336 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.182203    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cczmq\" (UniqueName: \"kubernetes.io/projected/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-kube-api-access-cczmq\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.283053 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.282965    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-tls\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.283205 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.283050    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cczmq\" (UniqueName: \"kubernetes.io/projected/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-kube-api-access-cczmq\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.283205 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.283094    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.283205 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.283139    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-grpc-tls\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.283205 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.283174    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.283401 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.283208    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-metrics-client-ca\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.283401 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.283243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.283401 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.283267    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.285706 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.285370    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-metrics-client-ca\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.287632 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.287606    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-grpc-tls\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.288489 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.288465    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.288595 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.288564    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.289209 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.289160    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-tls\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.289427 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.289363    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.290499 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.290392    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.301099 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.301053    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cczmq\" (UniqueName: \"kubernetes.io/projected/d00bf790-d0db-4f65-9a4d-2d17f7edd1db-kube-api-access-cczmq\") pod \"thanos-querier-db8557d74-dcchm\" (UID: \"d00bf790-d0db-4f65-9a4d-2d17f7edd1db\") " pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.350285 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.350250    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:01.570317 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:01.570229    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-db8557d74-dcchm"]
Jul 22 15:06:01.728598 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:06:01.728508    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00bf790_d0db_4f65_9a4d_2d17f7edd1db.slice/crio-5db9e250869e10ab35bd7cc39b83d8464d2da5c4d6749c295451e5a892930c1d WatchSource:0}: Error finding container 5db9e250869e10ab35bd7cc39b83d8464d2da5c4d6749c295451e5a892930c1d: Status 404 returned error can't find the container with id 5db9e250869e10ab35bd7cc39b83d8464d2da5c4d6749c295451e5a892930c1d
Jul 22 15:06:02.088888 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:02.088727    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm" event={"ID":"d00bf790-d0db-4f65-9a4d-2d17f7edd1db","Type":"ContainerStarted","Data":"5db9e250869e10ab35bd7cc39b83d8464d2da5c4d6749c295451e5a892930c1d"}
Jul 22 15:06:02.092328 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:02.092297    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tvwsg" event={"ID":"251ab756-226a-491c-ab81-d021165a4738","Type":"ContainerStarted","Data":"b5438dd81096c9cb6e5c3cbf4acf93db8340fb6f2a2564e2e23d74c9ae107043"}
Jul 22 15:06:02.941371 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:02.941335    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8ckgj"
Jul 22 15:06:05.107799 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.107719    2560 generic.go:356] "Generic (PLEG): container finished" podID="251ab756-226a-491c-ab81-d021165a4738" containerID="b5438dd81096c9cb6e5c3cbf4acf93db8340fb6f2a2564e2e23d74c9ae107043" exitCode=0
Jul 22 15:06:05.108222 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.107796    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tvwsg" event={"ID":"251ab756-226a-491c-ab81-d021165a4738","Type":"ContainerDied","Data":"b5438dd81096c9cb6e5c3cbf4acf93db8340fb6f2a2564e2e23d74c9ae107043"}
Jul 22 15:06:05.110744 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.110703    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbrrl" event={"ID":"b713f6e2-248e-4563-b2c3-682c1fc41469","Type":"ContainerStarted","Data":"9f2098ae2c84ccc5d15f0712a0c5f91a133af21d83f086ef90f3c65e29120368"}
Jul 22 15:06:05.151021 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.150979    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pbrrl" podStartSLOduration=1.13099745 podStartE2EDuration="29.150961169s" totalImagesPullingTime="28.019963719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:05:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:06:05.149868776 +0000 UTC m=+103.604582418" watchObservedRunningTime="2026-07-22 15:06:05.150961169 +0000 UTC m=+103.605674785"
Jul 22 15:06:05.335969 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.335930    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:06:05.366826 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.366767    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.371554 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.371530    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 15:06:05.371817 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.371802    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 15:06:05.372538 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.372516    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 15:06:05.373365 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.373033    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 15:06:05.373365 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.373239    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 15:06:05.373365 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.373320    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4rlb7maim8jmr\""
Jul 22 15:06:05.373749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.373734    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 15:06:05.373847 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.373833    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 15:06:05.374406 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.374389    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 15:06:05.374512 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.374401    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 15:06:05.374827 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.374799    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 15:06:05.375677 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.375658    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 15:06:05.375677 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.375670    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-crdsw\""
Jul 22 15:06:05.375838 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.375670    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 15:06:05.388812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.388131    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 15:06:05.394139 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.394113    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 15:06:05.397260 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.397237    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 15:06:05.399337 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.399303    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:06:05.530501 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.530670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530533    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.530670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530575    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.530670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530617    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.530670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530650    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.530900 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530693    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.530900 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530717    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.530900 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530778    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.530900 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530811    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.530900 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531143 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531143 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.530976    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531143 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.531019    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531143 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.531061    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531143 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.531091    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531143 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.531114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hgzt\" (UniqueName: \"kubernetes.io/projected/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-kube-api-access-8hgzt\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531143 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.531137    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-config\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531411 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.531164    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531411 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.531259    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.531411 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.531305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.632730 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.632656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.632730 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.632695    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hgzt\" (UniqueName: \"kubernetes.io/projected/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-kube-api-access-8hgzt\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.632730 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.632713    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-config\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.632996 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.632737    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.632996 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.632766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.632996 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.632951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633138 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633015    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633138 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633049    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633138 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633073    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633138 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633104    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633138 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633134    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633146    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633186    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633265    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633328    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633391 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633392    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.633847 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.633442    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.634690 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.634315    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.634690 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.634687    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.634881 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.634747    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.634944 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.634929    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.636020 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.635276    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.636020 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.635980    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.636873 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.636852    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.637033 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.636950    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-config\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.637889 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.637863    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.638228 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.638202    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.638823 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.638801    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.638911 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.638824    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.638911 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.638830    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.639301 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.639279    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.639301 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.639294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.639529 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.639506    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.639646 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.639627    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.640880 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.640836    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.642363 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.642339    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hgzt\" (UniqueName: \"kubernetes.io/projected/fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea-kube-api-access-8hgzt\") pod \"prometheus-k8s-0\" (UID: \"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:05.677004 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:05.676974    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:06.117851 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:06.117811    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tvwsg" event={"ID":"251ab756-226a-491c-ab81-d021165a4738","Type":"ContainerStarted","Data":"b28face7f9546b677622cbe42e15745fded337d6539cf8db2cea34dd4a4e2f5c"}
Jul 22 15:06:06.118660 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:06.117882    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tvwsg" event={"ID":"251ab756-226a-491c-ab81-d021165a4738","Type":"ContainerStarted","Data":"fee15d19d90edd3585e305013aa435e8c7468309a6a737af2cab1bea28118534"}
Jul 22 15:06:06.119574 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:06.119547    2560 generic.go:356] "Generic (PLEG): container finished" podID="7096c167-bdc7-408b-acea-15439fe7b559" containerID="30112be0634e346ef2397ce288843df3668edfa82d39f118712c940cc760ff12" exitCode=0
Jul 22 15:06:06.119698 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:06.119625    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7096c167-bdc7-408b-acea-15439fe7b559","Type":"ContainerDied","Data":"30112be0634e346ef2397ce288843df3668edfa82d39f118712c940cc760ff12"}
Jul 22 15:06:06.138532 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:06.138482    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tvwsg" podStartSLOduration=6.249285346 podStartE2EDuration="8.13846301s" totalImagesPullingTime="1.773406552s" totalInitContainerRuntime="115.771112ms" isStatefulPod=true podCreationTimestamp="2026-07-22 15:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:06:06.137336148 +0000 UTC m=+104.592049798" watchObservedRunningTime="2026-07-22 15:06:06.13846301 +0000 UTC m=+104.593176616"
Jul 22 15:06:06.305792 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:06.305769    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 15:06:06.307398 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:06:06.307359    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8eb493_f4b1_4be7_8311_7c59d3e9a2ea.slice/crio-d113f1d69485524a8262929c5bb1304bc8710a01f7cf69c56b7b14b38bef954e WatchSource:0}: Error finding container d113f1d69485524a8262929c5bb1304bc8710a01f7cf69c56b7b14b38bef954e: Status 404 returned error can't find the container with id d113f1d69485524a8262929c5bb1304bc8710a01f7cf69c56b7b14b38bef954e
Jul 22 15:06:07.127343 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:07.127262    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm" event={"ID":"d00bf790-d0db-4f65-9a4d-2d17f7edd1db","Type":"ContainerStarted","Data":"493500130a1b9bffecfc39e071c9559f7f1ac48c6c674be3d97825ad17985eb9"}
Jul 22 15:06:07.127829 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:07.127348    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm" event={"ID":"d00bf790-d0db-4f65-9a4d-2d17f7edd1db","Type":"ContainerStarted","Data":"7e39c2c380d23a5f60e62bb3f00f4ac87774f0b2daa46f1ad70fceb8a60fe23f"}
Jul 22 15:06:07.127829 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:07.127386    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm" event={"ID":"d00bf790-d0db-4f65-9a4d-2d17f7edd1db","Type":"ContainerStarted","Data":"d4866b41aa908f6b9def07dd3c9b8ac1af5983b89afb83b712e41a3866b5a574"}
Jul 22 15:06:07.129168 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:07.129130    2560 generic.go:356] "Generic (PLEG): container finished" podID="fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea" containerID="a700c66910b0fb97bbdde85a2808b19458c8c79c5f5e67c98cfde01e19d983e3" exitCode=0
Jul 22 15:06:07.129300 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:07.129163    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea","Type":"ContainerDied","Data":"a700c66910b0fb97bbdde85a2808b19458c8c79c5f5e67c98cfde01e19d983e3"}
Jul 22 15:06:07.129300 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:07.129232    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea","Type":"ContainerStarted","Data":"d113f1d69485524a8262929c5bb1304bc8710a01f7cf69c56b7b14b38bef954e"}
Jul 22 15:06:09.142084 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.142049    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm" event={"ID":"d00bf790-d0db-4f65-9a4d-2d17f7edd1db","Type":"ContainerStarted","Data":"e9a13b9d31a10f8229cde1eb12f2242bcd4128973d26244b4fc57b6ac9cc3d02"}
Jul 22 15:06:09.142523 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.142143    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm" event={"ID":"d00bf790-d0db-4f65-9a4d-2d17f7edd1db","Type":"ContainerStarted","Data":"e25699b1b81e811e4bf6fbf96e155ddad345a64aec593a9cb1147a24b7c30d8e"}
Jul 22 15:06:09.142523 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.142189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm" event={"ID":"d00bf790-d0db-4f65-9a4d-2d17f7edd1db","Type":"ContainerStarted","Data":"0811f92d73cf5fd7198d871da8755f0123ec0b21bfc8678c35a76ee848be7359"}
Jul 22 15:06:09.142523 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.142237    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:09.145325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.145302    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7096c167-bdc7-408b-acea-15439fe7b559","Type":"ContainerStarted","Data":"13d16272a9752625987e28dec5ea913df2aba88d0159c962ef2bb6d0f01c6ec7"}
Jul 22 15:06:09.145421 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.145372    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7096c167-bdc7-408b-acea-15439fe7b559","Type":"ContainerStarted","Data":"1ce095196cd253e7aee5ef598d2413aff140d90683183e1b805afc2da0c649e0"}
Jul 22 15:06:09.145588 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.145504    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7096c167-bdc7-408b-acea-15439fe7b559","Type":"ContainerStarted","Data":"733b93c224e9bfacc5f0fca2a08665a6b0e19c2ed479047e012262a0e88d5f7f"}
Jul 22 15:06:09.145588 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.145568    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7096c167-bdc7-408b-acea-15439fe7b559","Type":"ContainerStarted","Data":"ecf6c363552b6eae6e827fbc470c8d097a5253caefe497fe90bcb6258f25c875"}
Jul 22 15:06:09.145720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.145606    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7096c167-bdc7-408b-acea-15439fe7b559","Type":"ContainerStarted","Data":"846d03c98ba22fad2d1247b1ab28ddc38d847318cf99ca139114a0c4a1ca1805"}
Jul 22 15:06:09.175932 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:09.175892    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm" podStartSLOduration=2.7634825640000003 podStartE2EDuration="9.175876553s" totalImagesPullingTime="6.412393989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:06:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:06:09.174472607 +0000 UTC m=+107.629186240" watchObservedRunningTime="2026-07-22 15:06:09.175876553 +0000 UTC m=+107.630590167"
Jul 22 15:06:10.153907 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:10.153864    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7096c167-bdc7-408b-acea-15439fe7b559","Type":"ContainerStarted","Data":"3c48367fa3fe4a0d31f4ebfdc7593f221481bdd3c45892e12698c5a2e33bc004"}
Jul 22 15:06:10.187394 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:10.187343    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.390498031 podStartE2EDuration="10.187326161s" totalImagesPullingTime="6.767538523s" totalInitContainerRuntime="29.289607ms" isStatefulPod=false podCreationTimestamp="2026-07-22 15:06:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:06:10.185774361 +0000 UTC m=+108.640487975" watchObservedRunningTime="2026-07-22 15:06:10.187326161 +0000 UTC m=+108.642039776"
Jul 22 15:06:12.163096 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:12.163062    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea","Type":"ContainerStarted","Data":"38741c9d45f955e0d76ac1c8dbb236ac6e8a3884feb8091fb7e1e5c4c53f3a86"}
Jul 22 15:06:13.170179 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:13.170141    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea","Type":"ContainerStarted","Data":"daa93f04f2cbb5e72d047f3b59195042ca26d8b2e9310133acff92a162ba95b9"}
Jul 22 15:06:13.170553 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:13.170262    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea","Type":"ContainerStarted","Data":"b230ad8f4867c77f461edde85a5aa034c9918315379bb413d299375393809af1"}
Jul 22 15:06:13.170553 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:13.170307    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea","Type":"ContainerStarted","Data":"293c7c8b81cd79dc1e3d82742c26c29704d24add48488767ff40b8850d4130e6"}
Jul 22 15:06:13.170553 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:13.170346    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea","Type":"ContainerStarted","Data":"dece47cb7d93bc7d995bc13fa473086d6cca40b6ebde35f3613a5bc2fdc8710e"}
Jul 22 15:06:13.170553 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:13.170384    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea","Type":"ContainerStarted","Data":"16c712ff4e458616cf309f66a667a6ba3f628e2a0c12238c6bdff4dd174c9336"}
Jul 22 15:06:13.241644 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:13.241596    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.401118809 podStartE2EDuration="8.241581611s" totalImagesPullingTime="4.823530254s" totalInitContainerRuntime="16.932548ms" isStatefulPod=false podCreationTimestamp="2026-07-22 15:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:06:13.241144867 +0000 UTC m=+111.695858496" watchObservedRunningTime="2026-07-22 15:06:13.241581611 +0000 UTC m=+111.696295225"
Jul 22 15:06:15.160778 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:15.160749    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-db8557d74-dcchm"
Jul 22 15:06:15.677597 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:15.677568    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:06:21.204781 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:21.204746    2560 generic.go:356] "Generic (PLEG): container finished" podID="880ce076-2799-46e2-b8d2-950e9f74dee0" containerID="de8f43a3e3ea4dff3abab4f7ab7fddbdb68067d642a64cc1af509f5b46bb2061" exitCode=0
Jul 22 15:06:21.205244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:21.204798    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77" event={"ID":"880ce076-2799-46e2-b8d2-950e9f74dee0","Type":"ContainerDied","Data":"de8f43a3e3ea4dff3abab4f7ab7fddbdb68067d642a64cc1af509f5b46bb2061"}
Jul 22 15:06:21.205244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:21.205159    2560 scope.go:117] "RemoveContainer" containerID="de8f43a3e3ea4dff3abab4f7ab7fddbdb68067d642a64cc1af509f5b46bb2061"
Jul 22 15:06:22.210495 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:22.210459    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4qt77" event={"ID":"880ce076-2799-46e2-b8d2-950e9f74dee0","Type":"ContainerStarted","Data":"db3ffea85bbdaa6aa1e0ef35a860cd94a5c44378e5290851308508fa7f8ee194"}
Jul 22 15:06:24.201769 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.201724    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8" podUID="b9720254-d650-4e81-9417-3355eb50cce1" containerName="registry" containerID="cri-o://d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f" gracePeriod=30
Jul 22 15:06:24.495171 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.495142    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:06:24.596141 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596111    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b9720254-d650-4e81-9417-3355eb50cce1-ca-trust-extracted\") pod \"b9720254-d650-4e81-9417-3355eb50cce1\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") "
Jul 22 15:06:24.596141 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596145    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-bound-sa-token\") pod \"b9720254-d650-4e81-9417-3355eb50cce1\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") "
Jul 22 15:06:24.596338 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596170    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-registry-certificates\") pod \"b9720254-d650-4e81-9417-3355eb50cce1\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") "
Jul 22 15:06:24.596338 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596221    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-installation-pull-secrets\") pod \"b9720254-d650-4e81-9417-3355eb50cce1\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") "
Jul 22 15:06:24.596338 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596240    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjjj5\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-kube-api-access-hjjj5\") pod \"b9720254-d650-4e81-9417-3355eb50cce1\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") "
Jul 22 15:06:24.596338 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596266    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") pod \"b9720254-d650-4e81-9417-3355eb50cce1\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") "
Jul 22 15:06:24.596338 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596310    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-image-registry-private-configuration\") pod \"b9720254-d650-4e81-9417-3355eb50cce1\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") "
Jul 22 15:06:24.596338 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596339    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-trusted-ca\") pod \"b9720254-d650-4e81-9417-3355eb50cce1\" (UID: \"b9720254-d650-4e81-9417-3355eb50cce1\") "
Jul 22 15:06:24.596678 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596634    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b9720254-d650-4e81-9417-3355eb50cce1" (UID: "b9720254-d650-4e81-9417-3355eb50cce1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:06:24.597020 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.596988    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b9720254-d650-4e81-9417-3355eb50cce1" (UID: "b9720254-d650-4e81-9417-3355eb50cce1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:06:24.599213 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.599169    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b9720254-d650-4e81-9417-3355eb50cce1" (UID: "b9720254-d650-4e81-9417-3355eb50cce1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:06:24.599325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.599224    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b9720254-d650-4e81-9417-3355eb50cce1" (UID: "b9720254-d650-4e81-9417-3355eb50cce1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:06:24.599325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.599309    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b9720254-d650-4e81-9417-3355eb50cce1" (UID: "b9720254-d650-4e81-9417-3355eb50cce1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:06:24.599431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.599360    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b9720254-d650-4e81-9417-3355eb50cce1" (UID: "b9720254-d650-4e81-9417-3355eb50cce1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:06:24.599495 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.599467    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-kube-api-access-hjjj5" (OuterVolumeSpecName: "kube-api-access-hjjj5") pod "b9720254-d650-4e81-9417-3355eb50cce1" (UID: "b9720254-d650-4e81-9417-3355eb50cce1"). InnerVolumeSpecName "kube-api-access-hjjj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:06:24.605131 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.605105    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9720254-d650-4e81-9417-3355eb50cce1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b9720254-d650-4e81-9417-3355eb50cce1" (UID: "b9720254-d650-4e81-9417-3355eb50cce1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:06:24.697558 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.697525    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b9720254-d650-4e81-9417-3355eb50cce1-ca-trust-extracted\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:06:24.697558 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.697557    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-bound-sa-token\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:06:24.697766 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.697570    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-registry-certificates\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:06:24.697766 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.697584    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-installation-pull-secrets\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:06:24.697766 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.697595    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjjj5\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-kube-api-access-hjjj5\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:06:24.697766 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.697607    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b9720254-d650-4e81-9417-3355eb50cce1-registry-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:06:24.697766 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.697620    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b9720254-d650-4e81-9417-3355eb50cce1-image-registry-private-configuration\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:06:24.697766 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:24.697632    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b9720254-d650-4e81-9417-3355eb50cce1-trusted-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:06:25.227024 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:25.226990    2560 generic.go:356] "Generic (PLEG): container finished" podID="b9720254-d650-4e81-9417-3355eb50cce1" containerID="d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f" exitCode=0
Jul 22 15:06:25.227431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:25.227036    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8"
Jul 22 15:06:25.227431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:25.227051    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8" event={"ID":"b9720254-d650-4e81-9417-3355eb50cce1","Type":"ContainerDied","Data":"d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f"}
Jul 22 15:06:25.227431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:25.227092    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56df5f6fcb-jvxb8" event={"ID":"b9720254-d650-4e81-9417-3355eb50cce1","Type":"ContainerDied","Data":"cacf7f814108216fb69395d5d3c235948bd2983ad56452ff001d132f65a677ea"}
Jul 22 15:06:25.227431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:25.227110    2560 scope.go:117] "RemoveContainer" containerID="d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f"
Jul 22 15:06:25.237861 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:25.237841    2560 scope.go:117] "RemoveContainer" containerID="d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f"
Jul 22 15:06:25.238240 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:06:25.238219    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f\": container with ID starting with d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f not found: ID does not exist" containerID="d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f"
Jul 22 15:06:25.238303 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:25.238248    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f"} err="failed to get container status \"d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f\": rpc error: code = NotFound desc = could not find container \"d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f\": container with ID starting with d2b73749aebcb0dc12d1d017a7483ee405b1d7dc90eedde03fb5e66646933c2f not found: ID does not exist"
Jul 22 15:06:25.248996 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:25.248971    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56df5f6fcb-jvxb8"]
Jul 22 15:06:25.251998 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:25.251977    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56df5f6fcb-jvxb8"]
Jul 22 15:06:26.600745 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:06:26.600709    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9720254-d650-4e81-9417-3355eb50cce1" path="/var/lib/kubelet/pods/b9720254-d650-4e81-9417-3355eb50cce1/volumes"
Jul 22 15:07:05.677354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:07:05.677320    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:07:05.692861 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:07:05.692838    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:07:06.399638 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:07:06.399607    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 15:09:21.993221 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:21.993195    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:09:21.997938 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:21.997915    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:09:22.000367 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:22.000347    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:09:22.003477 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:22.003459    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 15:09:22.004695 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:22.004678    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:09:37.760084 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.760049    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"]
Jul 22 15:09:37.761585 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.760390    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9720254-d650-4e81-9417-3355eb50cce1" containerName="registry"
Jul 22 15:09:37.761585 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.760402    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9720254-d650-4e81-9417-3355eb50cce1" containerName="registry"
Jul 22 15:09:37.761585 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.760471    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9720254-d650-4e81-9417-3355eb50cce1" containerName="registry"
Jul 22 15:09:37.762383 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.762368    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.767525 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.767503    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 15:09:37.767637 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.767543    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 15:09:37.768578 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.768561    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-klfb9\""
Jul 22 15:09:37.770930 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.770910    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 15:09:37.772315 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.772298    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 15:09:37.791985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.791954    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"]
Jul 22 15:09:37.825803 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.825761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12c27eb5-ef0d-4914-8325-7a9735960e92-webhook-cert\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.825966 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.825866    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12c27eb5-ef0d-4914-8325-7a9735960e92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.825966 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.825932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12c27eb5-ef0d-4914-8325-7a9735960e92-tmp\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.825966 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.825959    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12c27eb5-ef0d-4914-8325-7a9735960e92-manifests\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.826069 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.825975    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pct6\" (UniqueName: \"kubernetes.io/projected/12c27eb5-ef0d-4914-8325-7a9735960e92-kube-api-access-2pct6\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.927170 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.927130    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12c27eb5-ef0d-4914-8325-7a9735960e92-tmp\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.927170 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.927176    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12c27eb5-ef0d-4914-8325-7a9735960e92-manifests\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.927380 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.927298    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pct6\" (UniqueName: \"kubernetes.io/projected/12c27eb5-ef0d-4914-8325-7a9735960e92-kube-api-access-2pct6\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.927380 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.927341    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12c27eb5-ef0d-4914-8325-7a9735960e92-webhook-cert\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.927504 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.927441    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12c27eb5-ef0d-4914-8325-7a9735960e92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.927650 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.927633    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12c27eb5-ef0d-4914-8325-7a9735960e92-manifests\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.929467 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.929431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12c27eb5-ef0d-4914-8325-7a9735960e92-tmp\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.929883 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.929860    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12c27eb5-ef0d-4914-8325-7a9735960e92-webhook-cert\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.929929 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.929860    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12c27eb5-ef0d-4914-8325-7a9735960e92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:37.938700 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:37.938675    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pct6\" (UniqueName: \"kubernetes.io/projected/12c27eb5-ef0d-4914-8325-7a9735960e92-kube-api-access-2pct6\") pod \"opendatahub-operator-controller-manager-7777b6b677-k5hm4\" (UID: \"12c27eb5-ef0d-4914-8325-7a9735960e92\") " pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:38.073505 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:38.073381    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:38.253638 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:38.253561    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"]
Jul 22 15:09:38.256464 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:09:38.256427    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12c27eb5_ef0d_4914_8325_7a9735960e92.slice/crio-6225146d0c656c44bb08c578ced799785489e2eacbabd1126ffb9a5893881eee WatchSource:0}: Error finding container 6225146d0c656c44bb08c578ced799785489e2eacbabd1126ffb9a5893881eee: Status 404 returned error can't find the container with id 6225146d0c656c44bb08c578ced799785489e2eacbabd1126ffb9a5893881eee
Jul 22 15:09:38.258224 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:38.258205    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:09:38.965267 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:38.965227    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4" event={"ID":"12c27eb5-ef0d-4914-8325-7a9735960e92","Type":"ContainerStarted","Data":"6225146d0c656c44bb08c578ced799785489e2eacbabd1126ffb9a5893881eee"}
Jul 22 15:09:41.979806 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:41.979770    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4" event={"ID":"12c27eb5-ef0d-4914-8325-7a9735960e92","Type":"ContainerStarted","Data":"fd26815765e65822eed1d90cf8655897c5116405ed2da8cf9f2107d5f44864fa"}
Jul 22 15:09:42.985052 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:42.985011    2560 generic.go:356] "Generic (PLEG): container finished" podID="12c27eb5-ef0d-4914-8325-7a9735960e92" containerID="fd26815765e65822eed1d90cf8655897c5116405ed2da8cf9f2107d5f44864fa" exitCode=0
Jul 22 15:09:42.985531 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:42.985094    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4" event={"ID":"12c27eb5-ef0d-4914-8325-7a9735960e92","Type":"ContainerDied","Data":"fd26815765e65822eed1d90cf8655897c5116405ed2da8cf9f2107d5f44864fa"}
Jul 22 15:09:43.990685 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:43.990647    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4" event={"ID":"12c27eb5-ef0d-4914-8325-7a9735960e92","Type":"ContainerStarted","Data":"49de3feaa8e56d36ee303f24de8de185828d767f46bdf10c8bda2473627143bd"}
Jul 22 15:09:43.991096 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:43.990748    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:54.997665 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:54.997635    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4"
Jul 22 15:09:55.026832 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:55.026757    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7777b6b677-k5hm4" podStartSLOduration=13.493358372 podStartE2EDuration="18.026742353s" totalImagesPullingTime="3.502211037s" totalInitContainerRuntime="1.031172944s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:09:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:09:44.062478424 +0000 UTC m=+322.517192056" watchObservedRunningTime="2026-07-22 15:09:55.026742353 +0000 UTC m=+333.481455968"
Jul 22 15:09:58.341485 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.341433    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59659ff488-g66vh"]
Jul 22 15:09:58.351195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.351169    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.354045 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.354008    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 15:09:58.354200 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.354076    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 15:09:58.368485 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.368437    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59659ff488-g66vh"]
Jul 22 15:09:58.510042 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.510010    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c56b0d2-1712-4c40-9636-310afe140663-tmp\") pod \"kube-auth-proxy-59659ff488-g66vh\" (UID: \"0c56b0d2-1712-4c40-9636-310afe140663\") " pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.510199 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.510067    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p5d5\" (UniqueName: \"kubernetes.io/projected/0c56b0d2-1712-4c40-9636-310afe140663-kube-api-access-2p5d5\") pod \"kube-auth-proxy-59659ff488-g66vh\" (UID: \"0c56b0d2-1712-4c40-9636-310afe140663\") " pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.510199 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.510163    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c56b0d2-1712-4c40-9636-310afe140663-tls-certs\") pod \"kube-auth-proxy-59659ff488-g66vh\" (UID: \"0c56b0d2-1712-4c40-9636-310afe140663\") " pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.610934 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.610853    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p5d5\" (UniqueName: \"kubernetes.io/projected/0c56b0d2-1712-4c40-9636-310afe140663-kube-api-access-2p5d5\") pod \"kube-auth-proxy-59659ff488-g66vh\" (UID: \"0c56b0d2-1712-4c40-9636-310afe140663\") " pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.610934 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.610908    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c56b0d2-1712-4c40-9636-310afe140663-tls-certs\") pod \"kube-auth-proxy-59659ff488-g66vh\" (UID: \"0c56b0d2-1712-4c40-9636-310afe140663\") " pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.611115 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.610962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c56b0d2-1712-4c40-9636-310afe140663-tmp\") pod \"kube-auth-proxy-59659ff488-g66vh\" (UID: \"0c56b0d2-1712-4c40-9636-310afe140663\") " pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.613367 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.613344    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c56b0d2-1712-4c40-9636-310afe140663-tmp\") pod \"kube-auth-proxy-59659ff488-g66vh\" (UID: \"0c56b0d2-1712-4c40-9636-310afe140663\") " pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.613439 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.613420    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c56b0d2-1712-4c40-9636-310afe140663-tls-certs\") pod \"kube-auth-proxy-59659ff488-g66vh\" (UID: \"0c56b0d2-1712-4c40-9636-310afe140663\") " pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.636917 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.636880    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p5d5\" (UniqueName: \"kubernetes.io/projected/0c56b0d2-1712-4c40-9636-310afe140663-kube-api-access-2p5d5\") pod \"kube-auth-proxy-59659ff488-g66vh\" (UID: \"0c56b0d2-1712-4c40-9636-310afe140663\") " pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.661349 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.661317    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh"
Jul 22 15:09:58.821354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:58.821317    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59659ff488-g66vh"]
Jul 22 15:09:58.838236 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:09:58.838201    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c56b0d2_1712_4c40_9636_310afe140663.slice/crio-d6d8a402d13f8e5d821730e59d62b7526eb4346649ea27f9fd0d17c2d1e17c2e WatchSource:0}: Error finding container d6d8a402d13f8e5d821730e59d62b7526eb4346649ea27f9fd0d17c2d1e17c2e: Status 404 returned error can't find the container with id d6d8a402d13f8e5d821730e59d62b7526eb4346649ea27f9fd0d17c2d1e17c2e
Jul 22 15:09:59.048942 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:09:59.048894    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh" event={"ID":"0c56b0d2-1712-4c40-9636-310afe140663","Type":"ContainerStarted","Data":"d6d8a402d13f8e5d821730e59d62b7526eb4346649ea27f9fd0d17c2d1e17c2e"}
Jul 22 15:10:00.757728 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.757682    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"]
Jul 22 15:10:00.761746 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.761720    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:00.766928 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.766888    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 22 15:10:00.767336 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.767191    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 22 15:10:00.767698 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.767571    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 22 15:10:00.768050 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.768029    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 15:10:00.768140 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.768030    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qvl7d\""
Jul 22 15:10:00.780753 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.780728    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"]
Jul 22 15:10:00.931769 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.931724    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e9b2d4-8813-4e55-aae0-460a2f628366-local-certs\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:00.931955 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.931779    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-token\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:00.931955 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.931813    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rt9w\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-kube-api-access-7rt9w\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:00.931955 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.931832    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:00.931955 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.931857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:00.931955 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.931925    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-cacerts\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:00.932270 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:00.931975    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.033275 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.033177    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.033275 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.033247    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e9b2d4-8813-4e55-aae0-460a2f628366-local-certs\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.033537 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.033300    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-token\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.033537 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.033380    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rt9w\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-kube-api-access-7rt9w\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.033537 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.033425    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.033537 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.033489    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.033749 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.033540    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-cacerts\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.034532 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.034506    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.036279 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.036253    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.036439 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.036417    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.036897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.036864    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e9b2d4-8813-4e55-aae0-460a2f628366-local-certs\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.037064 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.037042    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-cacerts\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.048934 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.048895    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rt9w\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-kube-api-access-7rt9w\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.049646 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.049622    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-token\") pod \"istiod-openshift-gateway-7574459656-b4hct\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.074984 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.074952    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:01.836071 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:01.836046    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"]
Jul 22 15:10:01.838719 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:10:01.838693    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e9b2d4_8813_4e55_aae0_460a2f628366.slice/crio-bc5e9d3910d5ad7762c9f2f1ac811665794b29d5b3ac303a83eb52303d8dd888 WatchSource:0}: Error finding container bc5e9d3910d5ad7762c9f2f1ac811665794b29d5b3ac303a83eb52303d8dd888: Status 404 returned error can't find the container with id bc5e9d3910d5ad7762c9f2f1ac811665794b29d5b3ac303a83eb52303d8dd888
Jul 22 15:10:02.067915 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:02.067818    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct" event={"ID":"a3e9b2d4-8813-4e55-aae0-460a2f628366","Type":"ContainerStarted","Data":"bc5e9d3910d5ad7762c9f2f1ac811665794b29d5b3ac303a83eb52303d8dd888"}
Jul 22 15:10:02.069373 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:02.069338    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh" event={"ID":"0c56b0d2-1712-4c40-9636-310afe140663","Type":"ContainerStarted","Data":"2f801dd0ba839afdc1fcbb78ba59496d464cfa732a5bc5b2563c523767554571"}
Jul 22 15:10:02.093683 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:02.093639    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59659ff488-g66vh" podStartSLOduration=1.192297922 podStartE2EDuration="4.093622373s" totalImagesPullingTime="2.901324451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:10:02.092567795 +0000 UTC m=+340.547281430" watchObservedRunningTime="2026-07-22 15:10:02.093622373 +0000 UTC m=+340.548336054"
Jul 22 15:10:05.409762 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:05.409724    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 15:10:05.410028 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:05.409796    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 15:10:06.091740 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:06.091706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct" event={"ID":"a3e9b2d4-8813-4e55-aae0-460a2f628366","Type":"ContainerStarted","Data":"10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa"}
Jul 22 15:10:06.091934 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:06.091867    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:06.093325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:06.093301    2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-b4hct container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 15:10:06.093471 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:06.093348    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct" podUID="a3e9b2d4-8813-4e55-aae0-460a2f628366" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 15:10:06.173851 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:06.173798    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct" podStartSLOduration=2.605132006 podStartE2EDuration="6.173782371s" totalImagesPullingTime="3.568650365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:10:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:10:06.170812908 +0000 UTC m=+344.625526524" watchObservedRunningTime="2026-07-22 15:10:06.173782371 +0000 UTC m=+344.628495964"
Jul 22 15:10:07.096884 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.096857    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:07.798852 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.798808    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"]
Jul 22 15:10:07.803514 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.803488    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:07.807464 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.807428    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 15:10:07.808513 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.808493    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 22 15:10:07.808625 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.808524    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 22 15:10:07.808625 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.808500    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 22 15:10:07.810901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.810882    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 15:10:07.811234 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.811219    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xl9pg\""
Jul 22 15:10:07.820334 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:07.820305    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"]
Jul 22 15:10:08.000191 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.000157    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-manager-config\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.000356 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.000215    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-cert\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.000356 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.000235    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqhnc\" (UniqueName: \"kubernetes.io/projected/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-kube-api-access-tqhnc\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.000356 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.000291    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-metrics-cert\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.100717 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.100640    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-cert\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.100717 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.100671    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqhnc\" (UniqueName: \"kubernetes.io/projected/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-kube-api-access-tqhnc\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.100717 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.100692    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-metrics-cert\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.101114 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.100757    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-manager-config\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.101414 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.101395    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-manager-config\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.103144 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.103122    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-cert\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.103262 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.103162    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-metrics-cert\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.114763 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.114736    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqhnc\" (UniqueName: \"kubernetes.io/projected/8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35-kube-api-access-tqhnc\") pod \"lws-controller-manager-79668d66cc-pxqtr\" (UID: \"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35\") " pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.413014 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.412978    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:08.556533 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:08.556500    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"]
Jul 22 15:10:08.559826 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:10:08.559798    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d86ce5a_a82b_4fdb_bae5_6ba4c44eec35.slice/crio-3c861cced933f4160f63c439b6c8d43468cd7e59159c98f0a090645c4d1cd421 WatchSource:0}: Error finding container 3c861cced933f4160f63c439b6c8d43468cd7e59159c98f0a090645c4d1cd421: Status 404 returned error can't find the container with id 3c861cced933f4160f63c439b6c8d43468cd7e59159c98f0a090645c4d1cd421
Jul 22 15:10:09.106966 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:09.106930    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr" event={"ID":"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35","Type":"ContainerStarted","Data":"3c861cced933f4160f63c439b6c8d43468cd7e59159c98f0a090645c4d1cd421"}
Jul 22 15:10:11.399537 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.399507    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"]
Jul 22 15:10:11.400039 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.399673    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct" podUID="a3e9b2d4-8813-4e55-aae0-460a2f628366" containerName="discovery" containerID="cri-o://10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa" gracePeriod=30
Jul 22 15:10:11.661745 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.661680    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:11.726361 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.726334    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-ca-configmap\") pod \"a3e9b2d4-8813-4e55-aae0-460a2f628366\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") "
Jul 22 15:10:11.726539 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.726368    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-token\") pod \"a3e9b2d4-8813-4e55-aae0-460a2f628366\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") "
Jul 22 15:10:11.726539 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.726386    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rt9w\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-kube-api-access-7rt9w\") pod \"a3e9b2d4-8813-4e55-aae0-460a2f628366\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") "
Jul 22 15:10:11.726539 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.726416    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-cacerts\") pod \"a3e9b2d4-8813-4e55-aae0-460a2f628366\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") "
Jul 22 15:10:11.726657 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.726640    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-dns-cert\") pod \"a3e9b2d4-8813-4e55-aae0-460a2f628366\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") "
Jul 22 15:10:11.726701 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.726679    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e9b2d4-8813-4e55-aae0-460a2f628366-local-certs\") pod \"a3e9b2d4-8813-4e55-aae0-460a2f628366\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") "
Jul 22 15:10:11.726764 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.726746    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-kubeconfig\") pod \"a3e9b2d4-8813-4e55-aae0-460a2f628366\" (UID: \"a3e9b2d4-8813-4e55-aae0-460a2f628366\") "
Jul 22 15:10:11.726824 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.726793    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "a3e9b2d4-8813-4e55-aae0-460a2f628366" (UID: "a3e9b2d4-8813-4e55-aae0-460a2f628366"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:10:11.727030 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.727010    2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-ca-configmap\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:10:11.728979 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.728947    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "a3e9b2d4-8813-4e55-aae0-460a2f628366" (UID: "a3e9b2d4-8813-4e55-aae0-460a2f628366"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:10:11.729080 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.729036    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e9b2d4-8813-4e55-aae0-460a2f628366-local-certs" (OuterVolumeSpecName: "local-certs") pod "a3e9b2d4-8813-4e55-aae0-460a2f628366" (UID: "a3e9b2d4-8813-4e55-aae0-460a2f628366"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:10:11.729080 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.729054    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-cacerts" (OuterVolumeSpecName: "cacerts") pod "a3e9b2d4-8813-4e55-aae0-460a2f628366" (UID: "a3e9b2d4-8813-4e55-aae0-460a2f628366"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:10:11.729201 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.729089    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "a3e9b2d4-8813-4e55-aae0-460a2f628366" (UID: "a3e9b2d4-8813-4e55-aae0-460a2f628366"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:10:11.729201 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.729141    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-kube-api-access-7rt9w" (OuterVolumeSpecName: "kube-api-access-7rt9w") pod "a3e9b2d4-8813-4e55-aae0-460a2f628366" (UID: "a3e9b2d4-8813-4e55-aae0-460a2f628366"). InnerVolumeSpecName "kube-api-access-7rt9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:10:11.729201 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.729141    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3e9b2d4-8813-4e55-aae0-460a2f628366" (UID: "a3e9b2d4-8813-4e55-aae0-460a2f628366"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:10:11.828064 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.828028    2560 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-kubeconfig\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:10:11.828064 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.828058    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-token\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:10:11.828064 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.828072    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rt9w\" (UniqueName: \"kubernetes.io/projected/a3e9b2d4-8813-4e55-aae0-460a2f628366-kube-api-access-7rt9w\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:10:11.828292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.828084    2560 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-cacerts\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:10:11.828292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.828096    2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a3e9b2d4-8813-4e55-aae0-460a2f628366-istio-csr-dns-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:10:11.828292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:11.828108    2560 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3e9b2d4-8813-4e55-aae0-460a2f628366-local-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:10:12.120194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.120109    2560 generic.go:356] "Generic (PLEG): container finished" podID="a3e9b2d4-8813-4e55-aae0-460a2f628366" containerID="10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa" exitCode=0
Jul 22 15:10:12.120194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.120161    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"
Jul 22 15:10:12.120194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.120170    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct" event={"ID":"a3e9b2d4-8813-4e55-aae0-460a2f628366","Type":"ContainerDied","Data":"10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa"}
Jul 22 15:10:12.120416 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.120213    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b4hct" event={"ID":"a3e9b2d4-8813-4e55-aae0-460a2f628366","Type":"ContainerDied","Data":"bc5e9d3910d5ad7762c9f2f1ac811665794b29d5b3ac303a83eb52303d8dd888"}
Jul 22 15:10:12.120416 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.120235    2560 scope.go:117] "RemoveContainer" containerID="10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa"
Jul 22 15:10:12.129809 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.129784    2560 scope.go:117] "RemoveContainer" containerID="10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa"
Jul 22 15:10:12.130131 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:10:12.130106    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa\": container with ID starting with 10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa not found: ID does not exist" containerID="10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa"
Jul 22 15:10:12.130180 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.130141    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa"} err="failed to get container status \"10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa\": rpc error: code = NotFound desc = could not find container \"10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa\": container with ID starting with 10801cffb35b3e352c4aa693bcacc0141b2e74a4c73b947b87bf8c7e3704bdfa not found: ID does not exist"
Jul 22 15:10:12.145255 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.145218    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"]
Jul 22 15:10:12.151259 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.151232    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b4hct"]
Jul 22 15:10:12.601938 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:12.601900    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3e9b2d4-8813-4e55-aae0-460a2f628366" path="/var/lib/kubelet/pods/a3e9b2d4-8813-4e55-aae0-460a2f628366/volumes"
Jul 22 15:10:17.146728 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:17.146695    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr" event={"ID":"8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35","Type":"ContainerStarted","Data":"2a9e7bda53ab8c20859935d1174daf718174d9817eddadc94fdd1fd221350f51"}
Jul 22 15:10:17.147093 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:17.146882    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:10:17.168902 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:17.168856    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr" podStartSLOduration=1.743897516 podStartE2EDuration="10.168837154s" totalImagesPullingTime="8.424939638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:10:17.166536771 +0000 UTC m=+355.621250386" watchObservedRunningTime="2026-07-22 15:10:17.168837154 +0000 UTC m=+355.623550771"
Jul 22 15:10:28.153507 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:10:28.153473    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-79668d66cc-pxqtr"
Jul 22 15:11:45.025969 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.025888    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"]
Jul 22 15:11:45.026404 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.026213    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e9b2d4-8813-4e55-aae0-460a2f628366" containerName="discovery"
Jul 22 15:11:45.026404 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.026224    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e9b2d4-8813-4e55-aae0-460a2f628366" containerName="discovery"
Jul 22 15:11:45.026404 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.026281    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e9b2d4-8813-4e55-aae0-460a2f628366" containerName="discovery"
Jul 22 15:11:45.029055 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.029033    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.031855 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.031826    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 22 15:11:45.031855 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.031848    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 22 15:11:45.032093 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.032075    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 15:11:45.032345 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.032320    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 15:11:45.033281 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.033263    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4gzh6\""
Jul 22 15:11:45.046835 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.046810    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"]
Jul 22 15:11:45.119775 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.119745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/30e63893-b387-47cd-b788-14b399146eed-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zvfdm\" (UID: \"30e63893-b387-47cd-b788-14b399146eed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.119924 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.119780    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cb5b\" (UniqueName: \"kubernetes.io/projected/30e63893-b387-47cd-b788-14b399146eed-kube-api-access-2cb5b\") pod \"kuadrant-console-plugin-66b5774985-zvfdm\" (UID: \"30e63893-b387-47cd-b788-14b399146eed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.119924 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.119910    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/30e63893-b387-47cd-b788-14b399146eed-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zvfdm\" (UID: \"30e63893-b387-47cd-b788-14b399146eed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.221023 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.220982    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/30e63893-b387-47cd-b788-14b399146eed-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zvfdm\" (UID: \"30e63893-b387-47cd-b788-14b399146eed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.221023 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.221030    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/30e63893-b387-47cd-b788-14b399146eed-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zvfdm\" (UID: \"30e63893-b387-47cd-b788-14b399146eed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.221261 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.221053    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cb5b\" (UniqueName: \"kubernetes.io/projected/30e63893-b387-47cd-b788-14b399146eed-kube-api-access-2cb5b\") pod \"kuadrant-console-plugin-66b5774985-zvfdm\" (UID: \"30e63893-b387-47cd-b788-14b399146eed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.221828 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.221798    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/30e63893-b387-47cd-b788-14b399146eed-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zvfdm\" (UID: \"30e63893-b387-47cd-b788-14b399146eed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.223704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.223676    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/30e63893-b387-47cd-b788-14b399146eed-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zvfdm\" (UID: \"30e63893-b387-47cd-b788-14b399146eed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.233308 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.233279    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cb5b\" (UniqueName: \"kubernetes.io/projected/30e63893-b387-47cd-b788-14b399146eed-kube-api-access-2cb5b\") pod \"kuadrant-console-plugin-66b5774985-zvfdm\" (UID: \"30e63893-b387-47cd-b788-14b399146eed\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.349283 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.349198    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"
Jul 22 15:11:45.488253 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.488220    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm"]
Jul 22 15:11:45.491270 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:11:45.491241    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30e63893_b387_47cd_b788_14b399146eed.slice/crio-2b16e6392124663517c00bd0823ae5fadc034dcb757b89c469903caecdcade16 WatchSource:0}: Error finding container 2b16e6392124663517c00bd0823ae5fadc034dcb757b89c469903caecdcade16: Status 404 returned error can't find the container with id 2b16e6392124663517c00bd0823ae5fadc034dcb757b89c469903caecdcade16
Jul 22 15:11:45.528483 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:11:45.528433    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm" event={"ID":"30e63893-b387-47cd-b788-14b399146eed","Type":"ContainerStarted","Data":"2b16e6392124663517c00bd0823ae5fadc034dcb757b89c469903caecdcade16"}
Jul 22 15:12:07.638438 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:07.638402    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm" event={"ID":"30e63893-b387-47cd-b788-14b399146eed","Type":"ContainerStarted","Data":"af49d63f331cd81959edfa6fcc63384253a2b623f838e654511622fa14d5a662"}
Jul 22 15:12:07.659011 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:07.658962    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zvfdm" podStartSLOduration=1.063923367 podStartE2EDuration="22.658948572s" totalImagesPullingTime="21.595025205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:12:07.657485706 +0000 UTC m=+466.112199321" watchObservedRunningTime="2026-07-22 15:12:07.658948572 +0000 UTC m=+466.113662186"
Jul 22 15:12:29.046112 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.046073    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"]
Jul 22 15:12:29.094331 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.094294    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"]
Jul 22 15:12:29.094525 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.094439    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:29.097077 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.097053    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 22 15:12:29.143350 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.143320    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"]
Jul 22 15:12:29.184961 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.184925    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d38aa47e-7bc1-495e-87e1-16bb736fdb73-config-file\") pod \"limitador-limitador-64c94fd64f-8wgvc\" (UID: \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:29.184961 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.184963    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv79l\" (UniqueName: \"kubernetes.io/projected/d38aa47e-7bc1-495e-87e1-16bb736fdb73-kube-api-access-xv79l\") pod \"limitador-limitador-64c94fd64f-8wgvc\" (UID: \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:29.286248 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.286214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d38aa47e-7bc1-495e-87e1-16bb736fdb73-config-file\") pod \"limitador-limitador-64c94fd64f-8wgvc\" (UID: \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:29.286248 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.286254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv79l\" (UniqueName: \"kubernetes.io/projected/d38aa47e-7bc1-495e-87e1-16bb736fdb73-kube-api-access-xv79l\") pod \"limitador-limitador-64c94fd64f-8wgvc\" (UID: \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:29.286881 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.286858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d38aa47e-7bc1-495e-87e1-16bb736fdb73-config-file\") pod \"limitador-limitador-64c94fd64f-8wgvc\" (UID: \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:29.295951 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.295926    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv79l\" (UniqueName: \"kubernetes.io/projected/d38aa47e-7bc1-495e-87e1-16bb736fdb73-kube-api-access-xv79l\") pod \"limitador-limitador-64c94fd64f-8wgvc\" (UID: \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:29.406384 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.406344    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:29.551388 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.551360    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"]
Jul 22 15:12:29.553948 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:12:29.553917    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38aa47e_7bc1_495e_87e1_16bb736fdb73.slice/crio-b76dd4baa83eaac68be9f9ecdf7b08df2a5db408fd745c521e3030842a074162 WatchSource:0}: Error finding container b76dd4baa83eaac68be9f9ecdf7b08df2a5db408fd745c521e3030842a074162: Status 404 returned error can't find the container with id b76dd4baa83eaac68be9f9ecdf7b08df2a5db408fd745c521e3030842a074162
Jul 22 15:12:29.727364 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:29.727270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc" event={"ID":"d38aa47e-7bc1-495e-87e1-16bb736fdb73","Type":"ContainerStarted","Data":"b76dd4baa83eaac68be9f9ecdf7b08df2a5db408fd745c521e3030842a074162"}
Jul 22 15:12:30.031476 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.031386    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s2jz"]
Jul 22 15:12:30.035927 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.035907    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9s2jz"
Jul 22 15:12:30.038591 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.038568    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-j8kmk\""
Jul 22 15:12:30.043791 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.043766    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s2jz"]
Jul 22 15:12:30.094351 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.094312    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rspkg\" (UniqueName: \"kubernetes.io/projected/e34b8265-a912-449e-a7e9-383b55bc030a-kube-api-access-rspkg\") pod \"authorino-75d556bd77-9s2jz\" (UID: \"e34b8265-a912-449e-a7e9-383b55bc030a\") " pod="kuadrant-system/authorino-75d556bd77-9s2jz"
Jul 22 15:12:30.195281 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.195243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rspkg\" (UniqueName: \"kubernetes.io/projected/e34b8265-a912-449e-a7e9-383b55bc030a-kube-api-access-rspkg\") pod \"authorino-75d556bd77-9s2jz\" (UID: \"e34b8265-a912-449e-a7e9-383b55bc030a\") " pod="kuadrant-system/authorino-75d556bd77-9s2jz"
Jul 22 15:12:30.204603 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.204571    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rspkg\" (UniqueName: \"kubernetes.io/projected/e34b8265-a912-449e-a7e9-383b55bc030a-kube-api-access-rspkg\") pod \"authorino-75d556bd77-9s2jz\" (UID: \"e34b8265-a912-449e-a7e9-383b55bc030a\") " pod="kuadrant-system/authorino-75d556bd77-9s2jz"
Jul 22 15:12:30.345890 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.345785    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9s2jz"
Jul 22 15:12:30.521115 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.521034    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s2jz"]
Jul 22 15:12:30.524964 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:12:30.524935    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode34b8265_a912_449e_a7e9_383b55bc030a.slice/crio-44546c24947a2664005a9b36097e0706fa6aed2b01e9eaceeb886ebbd277b38a WatchSource:0}: Error finding container 44546c24947a2664005a9b36097e0706fa6aed2b01e9eaceeb886ebbd277b38a: Status 404 returned error can't find the container with id 44546c24947a2664005a9b36097e0706fa6aed2b01e9eaceeb886ebbd277b38a
Jul 22 15:12:30.733636 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:30.733593    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9s2jz" event={"ID":"e34b8265-a912-449e-a7e9-383b55bc030a","Type":"ContainerStarted","Data":"44546c24947a2664005a9b36097e0706fa6aed2b01e9eaceeb886ebbd277b38a"}
Jul 22 15:12:32.746266 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:32.746201    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc" event={"ID":"d38aa47e-7bc1-495e-87e1-16bb736fdb73","Type":"ContainerStarted","Data":"a7acd9e0b87ee75fcf016ad6f33a48f2b0aedb57cffcb2b06d354eb9a7edd633"}
Jul 22 15:12:32.746825 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:32.746464    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:32.763060 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:32.762968    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc" podStartSLOduration=0.80451155 podStartE2EDuration="3.76295214s" totalImagesPullingTime="2.95844059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:12:32.762316026 +0000 UTC m=+491.217029656" watchObservedRunningTime="2026-07-22 15:12:32.76295214 +0000 UTC m=+491.217665755"
Jul 22 15:12:34.758842 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:34.758799    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9s2jz" event={"ID":"e34b8265-a912-449e-a7e9-383b55bc030a","Type":"ContainerStarted","Data":"008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048"}
Jul 22 15:12:34.783670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:34.783624    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9s2jz" podStartSLOduration=0.71580162 podStartE2EDuration="4.783609186s" totalImagesPullingTime="4.067807566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:12:34.78209994 +0000 UTC m=+493.236813554" watchObservedRunningTime="2026-07-22 15:12:34.783609186 +0000 UTC m=+493.238322813"
Jul 22 15:12:43.753157 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:43.753124    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:44.013572 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:44.013493    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"]
Jul 22 15:12:44.013720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:44.013662    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc" podUID="d38aa47e-7bc1-495e-87e1-16bb736fdb73" containerName="limitador" containerID="cri-o://a7acd9e0b87ee75fcf016ad6f33a48f2b0aedb57cffcb2b06d354eb9a7edd633" gracePeriod=30
Jul 22 15:12:44.802708 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:44.802677    2560 generic.go:356] "Generic (PLEG): container finished" podID="d38aa47e-7bc1-495e-87e1-16bb736fdb73" containerID="a7acd9e0b87ee75fcf016ad6f33a48f2b0aedb57cffcb2b06d354eb9a7edd633" exitCode=0
Jul 22 15:12:44.803118 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:44.802751    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc" event={"ID":"d38aa47e-7bc1-495e-87e1-16bb736fdb73","Type":"ContainerDied","Data":"a7acd9e0b87ee75fcf016ad6f33a48f2b0aedb57cffcb2b06d354eb9a7edd633"}
Jul 22 15:12:44.956846 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:44.956820    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:45.026505 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.026462    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d38aa47e-7bc1-495e-87e1-16bb736fdb73-config-file\") pod \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\" (UID: \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\") "
Jul 22 15:12:45.026709 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.026512    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv79l\" (UniqueName: \"kubernetes.io/projected/d38aa47e-7bc1-495e-87e1-16bb736fdb73-kube-api-access-xv79l\") pod \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\" (UID: \"d38aa47e-7bc1-495e-87e1-16bb736fdb73\") "
Jul 22 15:12:45.026810 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.026784    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d38aa47e-7bc1-495e-87e1-16bb736fdb73-config-file" (OuterVolumeSpecName: "config-file") pod "d38aa47e-7bc1-495e-87e1-16bb736fdb73" (UID: "d38aa47e-7bc1-495e-87e1-16bb736fdb73"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:12:45.028664 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.028644    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38aa47e-7bc1-495e-87e1-16bb736fdb73-kube-api-access-xv79l" (OuterVolumeSpecName: "kube-api-access-xv79l") pod "d38aa47e-7bc1-495e-87e1-16bb736fdb73" (UID: "d38aa47e-7bc1-495e-87e1-16bb736fdb73"). InnerVolumeSpecName "kube-api-access-xv79l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:12:45.127610 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.127551    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d38aa47e-7bc1-495e-87e1-16bb736fdb73-config-file\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:12:45.127610 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.127573    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv79l\" (UniqueName: \"kubernetes.io/projected/d38aa47e-7bc1-495e-87e1-16bb736fdb73-kube-api-access-xv79l\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:12:45.683429 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.683402    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"]
Jul 22 15:12:45.683747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.683734    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38aa47e-7bc1-495e-87e1-16bb736fdb73" containerName="limitador"
Jul 22 15:12:45.683792 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.683748    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38aa47e-7bc1-495e-87e1-16bb736fdb73" containerName="limitador"
Jul 22 15:12:45.683833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.683808    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38aa47e-7bc1-495e-87e1-16bb736fdb73" containerName="limitador"
Jul 22 15:12:45.686259 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.686240    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:45.689179 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.689157    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 22 15:12:45.689288 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.689171    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5kh59\""
Jul 22 15:12:45.689420 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.689406    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 15:12:45.690300 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.690280    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 15:12:45.700321 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.700301    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"]
Jul 22 15:12:45.808716 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.808693    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"
Jul 22 15:12:45.808716 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.808703    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8wgvc" event={"ID":"d38aa47e-7bc1-495e-87e1-16bb736fdb73","Type":"ContainerDied","Data":"b76dd4baa83eaac68be9f9ecdf7b08df2a5db408fd745c521e3030842a074162"}
Jul 22 15:12:45.809179 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.808763    2560 scope.go:117] "RemoveContainer" containerID="a7acd9e0b87ee75fcf016ad6f33a48f2b0aedb57cffcb2b06d354eb9a7edd633"
Jul 22 15:12:45.832874 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.832850    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/78790f73-5d7a-4a33-bae8-d92dcee21bb1-data\") pod \"postgres-69fb4594cf-jg2fb\" (UID: \"78790f73-5d7a-4a33-bae8-d92dcee21bb1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:45.832972 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.832895    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85qss\" (UniqueName: \"kubernetes.io/projected/78790f73-5d7a-4a33-bae8-d92dcee21bb1-kube-api-access-85qss\") pod \"postgres-69fb4594cf-jg2fb\" (UID: \"78790f73-5d7a-4a33-bae8-d92dcee21bb1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:45.833223 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.833205    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"]
Jul 22 15:12:45.838379 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.838360    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8wgvc"]
Jul 22 15:12:45.933493 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.933415    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85qss\" (UniqueName: \"kubernetes.io/projected/78790f73-5d7a-4a33-bae8-d92dcee21bb1-kube-api-access-85qss\") pod \"postgres-69fb4594cf-jg2fb\" (UID: \"78790f73-5d7a-4a33-bae8-d92dcee21bb1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:45.933596 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.933526    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/78790f73-5d7a-4a33-bae8-d92dcee21bb1-data\") pod \"postgres-69fb4594cf-jg2fb\" (UID: \"78790f73-5d7a-4a33-bae8-d92dcee21bb1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:45.933837 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.933822    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/78790f73-5d7a-4a33-bae8-d92dcee21bb1-data\") pod \"postgres-69fb4594cf-jg2fb\" (UID: \"78790f73-5d7a-4a33-bae8-d92dcee21bb1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:45.942385 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.942357    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85qss\" (UniqueName: \"kubernetes.io/projected/78790f73-5d7a-4a33-bae8-d92dcee21bb1-kube-api-access-85qss\") pod \"postgres-69fb4594cf-jg2fb\" (UID: \"78790f73-5d7a-4a33-bae8-d92dcee21bb1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:45.996949 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:45.996926    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:46.123664 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:46.123638    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"]
Jul 22 15:12:46.126458 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:12:46.126418    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78790f73_5d7a_4a33_bae8_d92dcee21bb1.slice/crio-1a7cb9c681f6651c467a81a3b170e65512f29c1df8fb06b379a87d85cd9074a6 WatchSource:0}: Error finding container 1a7cb9c681f6651c467a81a3b170e65512f29c1df8fb06b379a87d85cd9074a6: Status 404 returned error can't find the container with id 1a7cb9c681f6651c467a81a3b170e65512f29c1df8fb06b379a87d85cd9074a6
Jul 22 15:12:46.602224 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:46.602189    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38aa47e-7bc1-495e-87e1-16bb736fdb73" path="/var/lib/kubelet/pods/d38aa47e-7bc1-495e-87e1-16bb736fdb73/volumes"
Jul 22 15:12:46.818184 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:46.818106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb" event={"ID":"78790f73-5d7a-4a33-bae8-d92dcee21bb1","Type":"ContainerStarted","Data":"1a7cb9c681f6651c467a81a3b170e65512f29c1df8fb06b379a87d85cd9074a6"}
Jul 22 15:12:50.840985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:50.840955    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb" event={"ID":"78790f73-5d7a-4a33-bae8-d92dcee21bb1","Type":"ContainerStarted","Data":"5d106298d5dc8d083d9843f2fdec5c438fa273112a64a37a3a69d9bf36dc760f"}
Jul 22 15:12:50.841314 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:50.841078    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:50.860016 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:50.859975    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb" podStartSLOduration=1.241150691 podStartE2EDuration="5.859961036s" totalImagesPullingTime="4.618810345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:12:50.858529379 +0000 UTC m=+509.313242998" watchObservedRunningTime="2026-07-22 15:12:50.859961036 +0000 UTC m=+509.314674650"
Jul 22 15:12:56.875261 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:56.875232    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jg2fb"
Jul 22 15:12:57.442197 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.442164    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhgsw"]
Jul 22 15:12:57.445847 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.445825    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhgsw"
Jul 22 15:12:57.451815 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.451793    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhgsw"]
Jul 22 15:12:57.638770 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.638744    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqjgm\" (UniqueName: \"kubernetes.io/projected/d39c8398-3a4e-4ba3-9efb-7dc21580b694-kube-api-access-sqjgm\") pod \"authorino-5d4b67c446-mhgsw\" (UID: \"d39c8398-3a4e-4ba3-9efb-7dc21580b694\") " pod="kuadrant-system/authorino-5d4b67c446-mhgsw"
Jul 22 15:12:57.724240 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.724175    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhgsw"]
Jul 22 15:12:57.724463 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:12:57.724428    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sqjgm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-mhgsw" podUID="d39c8398-3a4e-4ba3-9efb-7dc21580b694"
Jul 22 15:12:57.739395 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.739365    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqjgm\" (UniqueName: \"kubernetes.io/projected/d39c8398-3a4e-4ba3-9efb-7dc21580b694-kube-api-access-sqjgm\") pod \"authorino-5d4b67c446-mhgsw\" (UID: \"d39c8398-3a4e-4ba3-9efb-7dc21580b694\") " pod="kuadrant-system/authorino-5d4b67c446-mhgsw"
Jul 22 15:12:57.748771 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.748744    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqjgm\" (UniqueName: \"kubernetes.io/projected/d39c8398-3a4e-4ba3-9efb-7dc21580b694-kube-api-access-sqjgm\") pod \"authorino-5d4b67c446-mhgsw\" (UID: \"d39c8398-3a4e-4ba3-9efb-7dc21580b694\") " pod="kuadrant-system/authorino-5d4b67c446-mhgsw"
Jul 22 15:12:57.754069 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.754047    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6cb9cdc6c5-hdqdv"]
Jul 22 15:12:57.756428 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.756411    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:12:57.759533 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.759518    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 22 15:12:57.767971 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.767946    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cb9cdc6c5-hdqdv"]
Jul 22 15:12:57.871718 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.871689    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhgsw"
Jul 22 15:12:57.879739 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.879716    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhgsw"
Jul 22 15:12:57.941897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.941872    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-tls-cert\") pod \"authorino-6cb9cdc6c5-hdqdv\" (UID: \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\") " pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:12:57.942009 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:57.941935    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jmkr\" (UniqueName: \"kubernetes.io/projected/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-kube-api-access-9jmkr\") pod \"authorino-6cb9cdc6c5-hdqdv\" (UID: \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\") " pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:12:58.042468 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.042402    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqjgm\" (UniqueName: \"kubernetes.io/projected/d39c8398-3a4e-4ba3-9efb-7dc21580b694-kube-api-access-sqjgm\") pod \"d39c8398-3a4e-4ba3-9efb-7dc21580b694\" (UID: \"d39c8398-3a4e-4ba3-9efb-7dc21580b694\") "
Jul 22 15:12:58.042566 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.042545    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jmkr\" (UniqueName: \"kubernetes.io/projected/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-kube-api-access-9jmkr\") pod \"authorino-6cb9cdc6c5-hdqdv\" (UID: \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\") " pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:12:58.042601 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.042594    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-tls-cert\") pod \"authorino-6cb9cdc6c5-hdqdv\" (UID: \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\") " pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:12:58.044478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.044433    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d39c8398-3a4e-4ba3-9efb-7dc21580b694-kube-api-access-sqjgm" (OuterVolumeSpecName: "kube-api-access-sqjgm") pod "d39c8398-3a4e-4ba3-9efb-7dc21580b694" (UID: "d39c8398-3a4e-4ba3-9efb-7dc21580b694"). InnerVolumeSpecName "kube-api-access-sqjgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:12:58.044859 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.044842    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-tls-cert\") pod \"authorino-6cb9cdc6c5-hdqdv\" (UID: \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\") " pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:12:58.050138 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.050115    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jmkr\" (UniqueName: \"kubernetes.io/projected/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-kube-api-access-9jmkr\") pod \"authorino-6cb9cdc6c5-hdqdv\" (UID: \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\") " pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:12:58.065924 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.065905    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:12:58.144104 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.144073    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqjgm\" (UniqueName: \"kubernetes.io/projected/d39c8398-3a4e-4ba3-9efb-7dc21580b694-kube-api-access-sqjgm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:12:58.188167 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.188139    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cb9cdc6c5-hdqdv"]
Jul 22 15:12:58.191606 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:12:58.191578    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b78dc66_842c_451a_a475_5ba7e0b5bd2e.slice/crio-6c627d0f846270b4755ba0e0424174fc9b8837f9ac1f81ce5f0d2c1695e72fa8 WatchSource:0}: Error finding container 6c627d0f846270b4755ba0e0424174fc9b8837f9ac1f81ce5f0d2c1695e72fa8: Status 404 returned error can't find the container with id 6c627d0f846270b4755ba0e0424174fc9b8837f9ac1f81ce5f0d2c1695e72fa8
Jul 22 15:12:58.877491 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.877397    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv" event={"ID":"5b78dc66-842c-451a-a475-5ba7e0b5bd2e","Type":"ContainerStarted","Data":"6c627d0f846270b4755ba0e0424174fc9b8837f9ac1f81ce5f0d2c1695e72fa8"}
Jul 22 15:12:58.877491 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.877416    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhgsw"
Jul 22 15:12:58.909716 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.909686    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhgsw"]
Jul 22 15:12:58.913846 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:58.913818    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhgsw"]
Jul 22 15:12:59.883644 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:59.883607    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv" event={"ID":"5b78dc66-842c-451a-a475-5ba7e0b5bd2e","Type":"ContainerStarted","Data":"fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005"}
Jul 22 15:12:59.927878 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:59.927817    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv" podStartSLOduration=2.036926444 podStartE2EDuration="2.927797917s" totalImagesPullingTime="890.871473ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:12:59.899574504 +0000 UTC m=+518.354288114" watchObservedRunningTime="2026-07-22 15:12:59.927797917 +0000 UTC m=+518.382511556"
Jul 22 15:12:59.929359 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:59.929326    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s2jz"]
Jul 22 15:12:59.929569 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:12:59.929547    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9s2jz" podUID="e34b8265-a912-449e-a7e9-383b55bc030a" containerName="authorino" containerID="cri-o://008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048" gracePeriod=30
Jul 22 15:13:00.231691 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.231654    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9s2jz"
Jul 22 15:13:00.256978 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.256951    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rspkg\" (UniqueName: \"kubernetes.io/projected/e34b8265-a912-449e-a7e9-383b55bc030a-kube-api-access-rspkg\") pod \"e34b8265-a912-449e-a7e9-383b55bc030a\" (UID: \"e34b8265-a912-449e-a7e9-383b55bc030a\") "
Jul 22 15:13:00.258879 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.258853    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e34b8265-a912-449e-a7e9-383b55bc030a-kube-api-access-rspkg" (OuterVolumeSpecName: "kube-api-access-rspkg") pod "e34b8265-a912-449e-a7e9-383b55bc030a" (UID: "e34b8265-a912-449e-a7e9-383b55bc030a"). InnerVolumeSpecName "kube-api-access-rspkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:13:00.357746 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.357707    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rspkg\" (UniqueName: \"kubernetes.io/projected/e34b8265-a912-449e-a7e9-383b55bc030a-kube-api-access-rspkg\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:13:00.601689 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.601621    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d39c8398-3a4e-4ba3-9efb-7dc21580b694" path="/var/lib/kubelet/pods/d39c8398-3a4e-4ba3-9efb-7dc21580b694/volumes"
Jul 22 15:13:00.889724 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.889648    2560 generic.go:356] "Generic (PLEG): container finished" podID="e34b8265-a912-449e-a7e9-383b55bc030a" containerID="008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048" exitCode=0
Jul 22 15:13:00.889724 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.889687    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9s2jz"
Jul 22 15:13:00.889936 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.889730    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9s2jz" event={"ID":"e34b8265-a912-449e-a7e9-383b55bc030a","Type":"ContainerDied","Data":"008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048"}
Jul 22 15:13:00.889936 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.889785    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9s2jz" event={"ID":"e34b8265-a912-449e-a7e9-383b55bc030a","Type":"ContainerDied","Data":"44546c24947a2664005a9b36097e0706fa6aed2b01e9eaceeb886ebbd277b38a"}
Jul 22 15:13:00.889936 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.889810    2560 scope.go:117] "RemoveContainer" containerID="008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048"
Jul 22 15:13:00.898029 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.898014    2560 scope.go:117] "RemoveContainer" containerID="008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048"
Jul 22 15:13:00.898257 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:13:00.898240    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048\": container with ID starting with 008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048 not found: ID does not exist" containerID="008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048"
Jul 22 15:13:00.898301 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.898264    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048"} err="failed to get container status \"008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048\": rpc error: code = NotFound desc = could not find container \"008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048\": container with ID starting with 008fdd0e81a96f95b11c0590e007700147342ca24dce48906af1c50aff0f0048 not found: ID does not exist"
Jul 22 15:13:00.905625 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.905601    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s2jz"]
Jul 22 15:13:00.911359 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:00.911340    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9s2jz"]
Jul 22 15:13:02.601748 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:02.601707    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e34b8265-a912-449e-a7e9-383b55bc030a" path="/var/lib/kubelet/pods/e34b8265-a912-449e-a7e9-383b55bc030a/volumes"
Jul 22 15:13:05.114413 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.114378    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6995947675-b9wjv"]
Jul 22 15:13:05.114785 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.114770    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34b8265-a912-449e-a7e9-383b55bc030a" containerName="authorino"
Jul 22 15:13:05.114828 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.114787    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34b8265-a912-449e-a7e9-383b55bc030a" containerName="authorino"
Jul 22 15:13:05.114861 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.114853    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34b8265-a912-449e-a7e9-383b55bc030a" containerName="authorino"
Jul 22 15:13:05.134044 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.134016    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6995947675-b9wjv"]
Jul 22 15:13:05.134195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.134118    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:05.136893 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.136872    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 15:13:05.138006 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.137992    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 15:13:05.138108 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.138031    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ff5sn\""
Jul 22 15:13:05.198428 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.198399    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379f490a-3ea9-4878-8f4c-2b5e9113aacc-webhook-cert\") pod \"maas-controller-6995947675-b9wjv\" (UID: \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\") " pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:05.198547 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.198476    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l9b6\" (UniqueName: \"kubernetes.io/projected/379f490a-3ea9-4878-8f4c-2b5e9113aacc-kube-api-access-2l9b6\") pod \"maas-controller-6995947675-b9wjv\" (UID: \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\") " pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:05.299556 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.299526    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379f490a-3ea9-4878-8f4c-2b5e9113aacc-webhook-cert\") pod \"maas-controller-6995947675-b9wjv\" (UID: \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\") " pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:05.299654 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.299585    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l9b6\" (UniqueName: \"kubernetes.io/projected/379f490a-3ea9-4878-8f4c-2b5e9113aacc-kube-api-access-2l9b6\") pod \"maas-controller-6995947675-b9wjv\" (UID: \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\") " pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:05.301892 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.301870    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379f490a-3ea9-4878-8f4c-2b5e9113aacc-webhook-cert\") pod \"maas-controller-6995947675-b9wjv\" (UID: \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\") " pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:05.307862 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.307833    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l9b6\" (UniqueName: \"kubernetes.io/projected/379f490a-3ea9-4878-8f4c-2b5e9113aacc-kube-api-access-2l9b6\") pod \"maas-controller-6995947675-b9wjv\" (UID: \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\") " pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:05.443829 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.443805    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:05.569372 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.569348    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6995947675-b9wjv"]
Jul 22 15:13:05.571738 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:13:05.571712    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379f490a_3ea9_4878_8f4c_2b5e9113aacc.slice/crio-ec6da10eee319472baf28c978177782cb5bb51cca974184ab674eedda3359436 WatchSource:0}: Error finding container ec6da10eee319472baf28c978177782cb5bb51cca974184ab674eedda3359436: Status 404 returned error can't find the container with id ec6da10eee319472baf28c978177782cb5bb51cca974184ab674eedda3359436
Jul 22 15:13:05.918488 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:05.918434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-b9wjv" event={"ID":"379f490a-3ea9-4878-8f4c-2b5e9113aacc","Type":"ContainerStarted","Data":"ec6da10eee319472baf28c978177782cb5bb51cca974184ab674eedda3359436"}
Jul 22 15:13:07.930929 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:07.930898    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-b9wjv" event={"ID":"379f490a-3ea9-4878-8f4c-2b5e9113aacc","Type":"ContainerStarted","Data":"95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470"}
Jul 22 15:13:07.931339 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:07.931078    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:07.973213 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:07.973168    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6995947675-b9wjv" podStartSLOduration=0.797058277 podStartE2EDuration="2.973153816s" totalImagesPullingTime="2.176095539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:13:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:07.972135269 +0000 UTC m=+526.426848885" watchObservedRunningTime="2026-07-22 15:13:07.973153816 +0000 UTC m=+526.427867442"
Jul 22 15:13:18.937617 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:18.937545    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:13:41.542225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.542190    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-4fldw"]
Jul 22 15:13:41.550357 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.550334    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:41.556101 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.556078    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 22 15:13:41.576347 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.576324    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-4fldw"]
Jul 22 15:13:41.593782 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.593760    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wdvfv"]
Jul 22 15:13:41.598003 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.597983    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:13:41.602423 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.602405    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cbbfk\""
Jul 22 15:13:41.616298 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.616272    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wdvfv"]
Jul 22 15:13:41.701937 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.701904    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw8js\" (UniqueName: \"kubernetes.io/projected/a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626-kube-api-access-xw8js\") pod \"payload-pre-processing-7677b774bb-4fldw\" (UID: \"a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:41.702074 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.701966    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d0303e1-7bbf-4a02-bf7e-714766501070-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wdvfv\" (UID: \"6d0303e1-7bbf-4a02-bf7e-714766501070\") " pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:13:41.702074 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.701988    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-4fldw\" (UID: \"a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:41.702160 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.702076    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmb66\" (UniqueName: \"kubernetes.io/projected/6d0303e1-7bbf-4a02-bf7e-714766501070-kube-api-access-lmb66\") pod \"payload-processing-5bf47cd988-wdvfv\" (UID: \"6d0303e1-7bbf-4a02-bf7e-714766501070\") " pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:13:41.802804 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.802734    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmb66\" (UniqueName: \"kubernetes.io/projected/6d0303e1-7bbf-4a02-bf7e-714766501070-kube-api-access-lmb66\") pod \"payload-processing-5bf47cd988-wdvfv\" (UID: \"6d0303e1-7bbf-4a02-bf7e-714766501070\") " pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:13:41.802804 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.802784    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8js\" (UniqueName: \"kubernetes.io/projected/a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626-kube-api-access-xw8js\") pod \"payload-pre-processing-7677b774bb-4fldw\" (UID: \"a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:41.803003 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.802860    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d0303e1-7bbf-4a02-bf7e-714766501070-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wdvfv\" (UID: \"6d0303e1-7bbf-4a02-bf7e-714766501070\") " pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:13:41.803003 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.802890    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-4fldw\" (UID: \"a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:41.803597 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.803578    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-4fldw\" (UID: \"a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:41.803826 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.803805    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d0303e1-7bbf-4a02-bf7e-714766501070-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wdvfv\" (UID: \"6d0303e1-7bbf-4a02-bf7e-714766501070\") " pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:13:41.811484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.811433    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmb66\" (UniqueName: \"kubernetes.io/projected/6d0303e1-7bbf-4a02-bf7e-714766501070-kube-api-access-lmb66\") pod \"payload-processing-5bf47cd988-wdvfv\" (UID: \"6d0303e1-7bbf-4a02-bf7e-714766501070\") " pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:13:41.811668 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.811649    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw8js\" (UniqueName: \"kubernetes.io/projected/a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626-kube-api-access-xw8js\") pod \"payload-pre-processing-7677b774bb-4fldw\" (UID: \"a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:41.860176 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.860149    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:41.907871 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:41.907841    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:13:42.013155 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:42.013127    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-4fldw"]
Jul 22 15:13:42.016613 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:13:42.016585    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99ae2b0_eb0b_4a6f_9ced_f7b6acc63626.slice/crio-0308bd8ed9cffa954da836bd4e190a81a61daa9d7aed58cdd69d6b8bedbd6832 WatchSource:0}: Error finding container 0308bd8ed9cffa954da836bd4e190a81a61daa9d7aed58cdd69d6b8bedbd6832: Status 404 returned error can't find the container with id 0308bd8ed9cffa954da836bd4e190a81a61daa9d7aed58cdd69d6b8bedbd6832
Jul 22 15:13:42.064745 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:42.064719    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wdvfv"]
Jul 22 15:13:42.066290 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:13:42.066268    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0303e1_7bbf_4a02_bf7e_714766501070.slice/crio-3f57326989decdf472776e0c9445a6f0af79bcdff18b46c911ec9d97b8e75638 WatchSource:0}: Error finding container 3f57326989decdf472776e0c9445a6f0af79bcdff18b46c911ec9d97b8e75638: Status 404 returned error can't find the container with id 3f57326989decdf472776e0c9445a6f0af79bcdff18b46c911ec9d97b8e75638
Jul 22 15:13:42.081503 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:42.081477    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv" event={"ID":"6d0303e1-7bbf-4a02-bf7e-714766501070","Type":"ContainerStarted","Data":"3f57326989decdf472776e0c9445a6f0af79bcdff18b46c911ec9d97b8e75638"}
Jul 22 15:13:42.082577 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:42.082557    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw" event={"ID":"a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626","Type":"ContainerStarted","Data":"0308bd8ed9cffa954da836bd4e190a81a61daa9d7aed58cdd69d6b8bedbd6832"}
Jul 22 15:13:45.101736 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:45.101689    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv" event={"ID":"6d0303e1-7bbf-4a02-bf7e-714766501070","Type":"ContainerStarted","Data":"fc3eaafae8bd92d1d81905e066c8bde0e1e72add6888b361f4cc3905fde49f40"}
Jul 22 15:13:45.102315 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:45.101893    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:13:45.103183 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:45.103161    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw" event={"ID":"a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626","Type":"ContainerStarted","Data":"6fb674629dc9876bef2d7cfa74b90d790ac853efb970b972a9e96a39a771b563"}
Jul 22 15:13:45.103291 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:45.103274    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:45.129959 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:45.129913    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv" podStartSLOduration=1.585819933 podStartE2EDuration="4.129898355s" totalImagesPullingTime="2.544078422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:45.128463653 +0000 UTC m=+563.583177265" watchObservedRunningTime="2026-07-22 15:13:45.129898355 +0000 UTC m=+563.584611970"
Jul 22 15:13:45.155769 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:45.155714    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw" podStartSLOduration=1.574382365 podStartE2EDuration="4.155696427s" totalImagesPullingTime="2.581314062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:13:45.15333878 +0000 UTC m=+563.608052394" watchObservedRunningTime="2026-07-22 15:13:45.155696427 +0000 UTC m=+563.610410042"
Jul 22 15:13:56.109102 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:56.109072    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-4fldw"
Jul 22 15:13:56.109531 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:13:56.109139    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-wdvfv"
Jul 22 15:14:19.475898 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.475867    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"]
Jul 22 15:14:19.483124 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.483103    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.486064 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.486047    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 15:14:19.486189 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.486138    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 15:14:19.487307 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.487282    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 22 15:14:19.487307 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.487286    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9hwbl\""
Jul 22 15:14:19.491347 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.491324    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"]
Jul 22 15:14:19.608428 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.608388    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.608649 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.608482    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.608649 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.608532    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.608649 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.608585    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8849e49c-3323-4f58-b841-94ff6aba3eeb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.608649 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.608610    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.608810 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.608672    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d75q\" (UniqueName: \"kubernetes.io/projected/8849e49c-3323-4f58-b841-94ff6aba3eeb-kube-api-access-5d75q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.709844 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.709803    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8849e49c-3323-4f58-b841-94ff6aba3eeb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.710019 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.709848    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.710019 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.709911    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d75q\" (UniqueName: \"kubernetes.io/projected/8849e49c-3323-4f58-b841-94ff6aba3eeb-kube-api-access-5d75q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.710019 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.709960    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.710184 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.710031    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.710184 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.710092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.710372 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.710345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.710442 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.710359    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.710442 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.710405    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.712103 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.712084    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8849e49c-3323-4f58-b841-94ff6aba3eeb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.712378 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.712358    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8849e49c-3323-4f58-b841-94ff6aba3eeb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.721365 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.721342    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d75q\" (UniqueName: \"kubernetes.io/projected/8849e49c-3323-4f58-b841-94ff6aba3eeb-kube-api-access-5d75q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9mjwm\" (UID: \"8849e49c-3323-4f58-b841-94ff6aba3eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.794739 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.794646    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:19.930414 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:19.930386    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"]
Jul 22 15:14:19.932480 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:14:19.932432    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8849e49c_3323_4f58_b841_94ff6aba3eeb.slice/crio-9e57ce350ca1214a6fc630f50ca149ceb89eb53ace522dd8e74a5f024a23c2ab WatchSource:0}: Error finding container 9e57ce350ca1214a6fc630f50ca149ceb89eb53ace522dd8e74a5f024a23c2ab: Status 404 returned error can't find the container with id 9e57ce350ca1214a6fc630f50ca149ceb89eb53ace522dd8e74a5f024a23c2ab
Jul 22 15:14:20.272867 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:20.272822    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm" event={"ID":"8849e49c-3323-4f58-b841-94ff6aba3eeb","Type":"ContainerStarted","Data":"9e57ce350ca1214a6fc630f50ca149ceb89eb53ace522dd8e74a5f024a23c2ab"}
Jul 22 15:14:22.027779 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:22.027696    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:14:22.028417 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:22.028397    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:14:22.034305 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:22.034283    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:14:22.035031 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:22.035011    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:14:22.283263 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:22.283143    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm" event={"ID":"8849e49c-3323-4f58-b841-94ff6aba3eeb","Type":"ContainerStarted","Data":"9e742e8eb661e899ac1debfb81ac61076be8c6bd7def5722bf93348cec2577fb"}
Jul 22 15:14:22.304295 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:22.304243    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm" podStartSLOduration=1.4777053549999999 podStartE2EDuration="3.30422837s" totalImagesPullingTime="1.826523015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:22.302303374 +0000 UTC m=+600.757016988" watchObservedRunningTime="2026-07-22 15:14:22.30422837 +0000 UTC m=+600.758941984"
Jul 22 15:14:29.795818 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:29.795777    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:29.796291 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:29.795848    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:29.808552 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:29.808516    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:30.330456 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:30.330423    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9mjwm"
Jul 22 15:14:35.076608 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.076572    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"]
Jul 22 15:14:35.080422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.080399    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.083165 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.083143    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 22 15:14:35.100259 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.100236    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"]
Jul 22 15:14:35.143532 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.143505    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.143635 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.143548    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.143689 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.143627    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtp6n\" (UniqueName: \"kubernetes.io/projected/cc6cdb3d-0269-4b19-8340-f6d2331b3717-kube-api-access-rtp6n\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.143756 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.143709    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.143756 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.143739    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc6cdb3d-0269-4b19-8340-f6d2331b3717-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.143833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.143760    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.244606 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.244578    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc6cdb3d-0269-4b19-8340-f6d2331b3717-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.244729 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.244615    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.244729 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.244653    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.244729 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.244678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.244729 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.244709    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtp6n\" (UniqueName: \"kubernetes.io/projected/cc6cdb3d-0269-4b19-8340-f6d2331b3717-kube-api-access-rtp6n\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.244933 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.244758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.245152 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.245125    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.245239 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.245125    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.245239 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.245173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.246963 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.246935    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc6cdb3d-0269-4b19-8340-f6d2331b3717-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.247153 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.247134    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc6cdb3d-0269-4b19-8340-f6d2331b3717-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.257119 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.257098    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtp6n\" (UniqueName: \"kubernetes.io/projected/cc6cdb3d-0269-4b19-8340-f6d2331b3717-kube-api-access-rtp6n\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-krrt8\" (UID: \"cc6cdb3d-0269-4b19-8340-f6d2331b3717\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.390443 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.390383    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:35.520911 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:35.520887    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"]
Jul 22 15:14:35.523593 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:14:35.523568    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6cdb3d_0269_4b19_8340_f6d2331b3717.slice/crio-3617a3891a4556582ae22ddeae3ccf50d17818cb83cfa6e4c8626caaebcabcec WatchSource:0}: Error finding container 3617a3891a4556582ae22ddeae3ccf50d17818cb83cfa6e4c8626caaebcabcec: Status 404 returned error can't find the container with id 3617a3891a4556582ae22ddeae3ccf50d17818cb83cfa6e4c8626caaebcabcec
Jul 22 15:14:36.347717 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:36.347687    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8" event={"ID":"cc6cdb3d-0269-4b19-8340-f6d2331b3717","Type":"ContainerStarted","Data":"3617a3891a4556582ae22ddeae3ccf50d17818cb83cfa6e4c8626caaebcabcec"}
Jul 22 15:14:38.362473 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:38.362407    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8" event={"ID":"cc6cdb3d-0269-4b19-8340-f6d2331b3717","Type":"ContainerStarted","Data":"42c2e1af64722b0f44990c97326b95fc2a70a9ca7fe45804d2dfbf26eea0091d"}
Jul 22 15:14:38.384525 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:38.384469    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8" podStartSLOduration=1.256742943 podStartE2EDuration="3.384435778s" totalImagesPullingTime="2.127692835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:14:38.382978805 +0000 UTC m=+616.837692411" watchObservedRunningTime="2026-07-22 15:14:38.384435778 +0000 UTC m=+616.839149392"
Jul 22 15:14:45.390939 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:45.390844    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:45.391376 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:45.390983    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:45.414677 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:45.414651    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:14:45.426125 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:14:45.426097    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-krrt8"
Jul 22 15:15:14.216405 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.216370    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6cb9cdc6c5-hdqdv"]
Jul 22 15:15:14.216888 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.216636    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv" podUID="5b78dc66-842c-451a-a475-5ba7e0b5bd2e" containerName="authorino" containerID="cri-o://fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005" gracePeriod=30
Jul 22 15:15:14.461237 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.461209    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:15:14.531210 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.531131    2560 generic.go:356] "Generic (PLEG): container finished" podID="5b78dc66-842c-451a-a475-5ba7e0b5bd2e" containerID="fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005" exitCode=0
Jul 22 15:15:14.531210 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.531167    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv"
Jul 22 15:15:14.531373 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.531212    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv" event={"ID":"5b78dc66-842c-451a-a475-5ba7e0b5bd2e","Type":"ContainerDied","Data":"fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005"}
Jul 22 15:15:14.531373 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.531253    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cb9cdc6c5-hdqdv" event={"ID":"5b78dc66-842c-451a-a475-5ba7e0b5bd2e","Type":"ContainerDied","Data":"6c627d0f846270b4755ba0e0424174fc9b8837f9ac1f81ce5f0d2c1695e72fa8"}
Jul 22 15:15:14.531373 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.531274    2560 scope.go:117] "RemoveContainer" containerID="fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005"
Jul 22 15:15:14.539763 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.539742    2560 scope.go:117] "RemoveContainer" containerID="fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005"
Jul 22 15:15:14.540016 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:15:14.539996    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005\": container with ID starting with fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005 not found: ID does not exist" containerID="fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005"
Jul 22 15:15:14.540060 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.540026    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005"} err="failed to get container status \"fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005\": rpc error: code = NotFound desc = could not find container \"fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005\": container with ID starting with fa83be4162177275e86fd76580df5d2ec5b4fb82910587f7b291333586358005 not found: ID does not exist"
Jul 22 15:15:14.580729 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.580710    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jmkr\" (UniqueName: \"kubernetes.io/projected/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-kube-api-access-9jmkr\") pod \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\" (UID: \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\") "
Jul 22 15:15:14.580825 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.580743    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-tls-cert\") pod \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\" (UID: \"5b78dc66-842c-451a-a475-5ba7e0b5bd2e\") "
Jul 22 15:15:14.582650 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.582621    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-kube-api-access-9jmkr" (OuterVolumeSpecName: "kube-api-access-9jmkr") pod "5b78dc66-842c-451a-a475-5ba7e0b5bd2e" (UID: "5b78dc66-842c-451a-a475-5ba7e0b5bd2e"). InnerVolumeSpecName "kube-api-access-9jmkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:15:14.590731 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.590709    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5b78dc66-842c-451a-a475-5ba7e0b5bd2e" (UID: "5b78dc66-842c-451a-a475-5ba7e0b5bd2e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:15:14.681985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.681953    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jmkr\" (UniqueName: \"kubernetes.io/projected/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-kube-api-access-9jmkr\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:15:14.681985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.681980    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b78dc66-842c-451a-a475-5ba7e0b5bd2e-tls-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:15:14.847187 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.847109    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6cb9cdc6c5-hdqdv"]
Jul 22 15:15:14.851538 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:14.851515    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6cb9cdc6c5-hdqdv"]
Jul 22 15:15:16.602049 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:15:16.602017    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b78dc66-842c-451a-a475-5ba7e0b5bd2e" path="/var/lib/kubelet/pods/5b78dc66-842c-451a-a475-5ba7e0b5bd2e/volumes"
Jul 22 15:16:01.265554 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.265517    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6995947675-b9wjv"]
Jul 22 15:16:01.266015 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.265734    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6995947675-b9wjv" podUID="379f490a-3ea9-4878-8f4c-2b5e9113aacc" containerName="manager" containerID="cri-o://95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470" gracePeriod=30
Jul 22 15:16:01.513459 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.513429    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:16:01.585491 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.585386    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379f490a-3ea9-4878-8f4c-2b5e9113aacc-webhook-cert\") pod \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\" (UID: \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\") "
Jul 22 15:16:01.585491 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.585427    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l9b6\" (UniqueName: \"kubernetes.io/projected/379f490a-3ea9-4878-8f4c-2b5e9113aacc-kube-api-access-2l9b6\") pod \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\" (UID: \"379f490a-3ea9-4878-8f4c-2b5e9113aacc\") "
Jul 22 15:16:01.587571 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.587542    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/379f490a-3ea9-4878-8f4c-2b5e9113aacc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "379f490a-3ea9-4878-8f4c-2b5e9113aacc" (UID: "379f490a-3ea9-4878-8f4c-2b5e9113aacc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:16:01.587706 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.587653    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/379f490a-3ea9-4878-8f4c-2b5e9113aacc-kube-api-access-2l9b6" (OuterVolumeSpecName: "kube-api-access-2l9b6") pod "379f490a-3ea9-4878-8f4c-2b5e9113aacc" (UID: "379f490a-3ea9-4878-8f4c-2b5e9113aacc"). InnerVolumeSpecName "kube-api-access-2l9b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:16:01.686312 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.686277    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/379f490a-3ea9-4878-8f4c-2b5e9113aacc-webhook-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:16:01.686312 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.686306    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l9b6\" (UniqueName: \"kubernetes.io/projected/379f490a-3ea9-4878-8f4c-2b5e9113aacc-kube-api-access-2l9b6\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:16:01.750215 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.750179    2560 generic.go:356] "Generic (PLEG): container finished" podID="379f490a-3ea9-4878-8f4c-2b5e9113aacc" containerID="95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470" exitCode=0
Jul 22 15:16:01.750364 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.750224    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-b9wjv"
Jul 22 15:16:01.750364 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.750268    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-b9wjv" event={"ID":"379f490a-3ea9-4878-8f4c-2b5e9113aacc","Type":"ContainerDied","Data":"95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470"}
Jul 22 15:16:01.750364 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.750314    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-b9wjv" event={"ID":"379f490a-3ea9-4878-8f4c-2b5e9113aacc","Type":"ContainerDied","Data":"ec6da10eee319472baf28c978177782cb5bb51cca974184ab674eedda3359436"}
Jul 22 15:16:01.750364 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.750335    2560 scope.go:117] "RemoveContainer" containerID="95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470"
Jul 22 15:16:01.759219 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.759202    2560 scope.go:117] "RemoveContainer" containerID="95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470"
Jul 22 15:16:01.759491 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:16:01.759473    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470\": container with ID starting with 95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470 not found: ID does not exist" containerID="95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470"
Jul 22 15:16:01.759541 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.759501    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470"} err="failed to get container status \"95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470\": rpc error: code = NotFound desc = could not find container \"95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470\": container with ID starting with 95e393bd2bc189292c03c8df3dd88d0f86dfa6cfd30193961c41da9f54ae1470 not found: ID does not exist"
Jul 22 15:16:01.771325 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.771298    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6995947675-b9wjv"]
Jul 22 15:16:01.775148 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:01.775128    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6995947675-b9wjv"]
Jul 22 15:16:02.607877 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:16:02.607846    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="379f490a-3ea9-4878-8f4c-2b5e9113aacc" path="/var/lib/kubelet/pods/379f490a-3ea9-4878-8f4c-2b5e9113aacc/volumes"
Jul 22 15:17:02.668720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.668682    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6995947675-clm5r"]
Jul 22 15:17:02.669285 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.669197    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b78dc66-842c-451a-a475-5ba7e0b5bd2e" containerName="authorino"
Jul 22 15:17:02.669285 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.669217    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b78dc66-842c-451a-a475-5ba7e0b5bd2e" containerName="authorino"
Jul 22 15:17:02.669285 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.669251    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379f490a-3ea9-4878-8f4c-2b5e9113aacc" containerName="manager"
Jul 22 15:17:02.669285 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.669261    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="379f490a-3ea9-4878-8f4c-2b5e9113aacc" containerName="manager"
Jul 22 15:17:02.669533 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.669356    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="379f490a-3ea9-4878-8f4c-2b5e9113aacc" containerName="manager"
Jul 22 15:17:02.669533 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.669368    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b78dc66-842c-451a-a475-5ba7e0b5bd2e" containerName="authorino"
Jul 22 15:17:02.672478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.672443    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:02.676373 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.676350    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 15:17:02.676496 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.676376    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 15:17:02.676496 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.676438    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ff5sn\""
Jul 22 15:17:02.681373 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.681353    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6995947675-clm5r"]
Jul 22 15:17:02.828847 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.828810    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwd5z\" (UniqueName: \"kubernetes.io/projected/3e3d6a6b-d4e4-4771-a06f-985a713da099-kube-api-access-kwd5z\") pod \"maas-controller-6995947675-clm5r\" (UID: \"3e3d6a6b-d4e4-4771-a06f-985a713da099\") " pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:02.829005 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.828874    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e3d6a6b-d4e4-4771-a06f-985a713da099-webhook-cert\") pod \"maas-controller-6995947675-clm5r\" (UID: \"3e3d6a6b-d4e4-4771-a06f-985a713da099\") " pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:02.929618 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.929532    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e3d6a6b-d4e4-4771-a06f-985a713da099-webhook-cert\") pod \"maas-controller-6995947675-clm5r\" (UID: \"3e3d6a6b-d4e4-4771-a06f-985a713da099\") " pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:02.929771 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.929670    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwd5z\" (UniqueName: \"kubernetes.io/projected/3e3d6a6b-d4e4-4771-a06f-985a713da099-kube-api-access-kwd5z\") pod \"maas-controller-6995947675-clm5r\" (UID: \"3e3d6a6b-d4e4-4771-a06f-985a713da099\") " pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:02.932152 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.932127    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e3d6a6b-d4e4-4771-a06f-985a713da099-webhook-cert\") pod \"maas-controller-6995947675-clm5r\" (UID: \"3e3d6a6b-d4e4-4771-a06f-985a713da099\") " pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:02.941623 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.941600    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwd5z\" (UniqueName: \"kubernetes.io/projected/3e3d6a6b-d4e4-4771-a06f-985a713da099-kube-api-access-kwd5z\") pod \"maas-controller-6995947675-clm5r\" (UID: \"3e3d6a6b-d4e4-4771-a06f-985a713da099\") " pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:02.982980 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:02.982947    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:03.331677 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:03.331561    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6995947675-clm5r"]
Jul 22 15:17:03.334484 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:17:03.334429    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3d6a6b_d4e4_4771_a06f_985a713da099.slice/crio-5fa31a008eaea9ae8c2f261773ee0d40f8652215b7230d3c85692e5e5692a7a5 WatchSource:0}: Error finding container 5fa31a008eaea9ae8c2f261773ee0d40f8652215b7230d3c85692e5e5692a7a5: Status 404 returned error can't find the container with id 5fa31a008eaea9ae8c2f261773ee0d40f8652215b7230d3c85692e5e5692a7a5
Jul 22 15:17:03.335905 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:03.335885    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:17:04.033981 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:04.033946    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-clm5r" event={"ID":"3e3d6a6b-d4e4-4771-a06f-985a713da099","Type":"ContainerStarted","Data":"ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc"}
Jul 22 15:17:04.034337 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:04.034003    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-clm5r" event={"ID":"3e3d6a6b-d4e4-4771-a06f-985a713da099","Type":"ContainerStarted","Data":"5fa31a008eaea9ae8c2f261773ee0d40f8652215b7230d3c85692e5e5692a7a5"}
Jul 22 15:17:04.034337 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:04.034030    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:04.057191 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:04.057138    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6995947675-clm5r" podStartSLOduration=1.522001379 podStartE2EDuration="2.057122264s" totalImagesPullingTime="535.120885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:17:04.055715459 +0000 UTC m=+762.510429075" watchObservedRunningTime="2026-07-22 15:17:04.057122264 +0000 UTC m=+762.511835878"
Jul 22 15:17:15.040240 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:15.040205    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:19.769189 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:19.769155    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6995947675-clm5r"]
Jul 22 15:17:19.769610 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:19.769336    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6995947675-clm5r" podUID="3e3d6a6b-d4e4-4771-a06f-985a713da099" containerName="manager" containerID="cri-o://ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc" gracePeriod=30
Jul 22 15:17:20.010739 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.010711    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:20.075815 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.075728    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwd5z\" (UniqueName: \"kubernetes.io/projected/3e3d6a6b-d4e4-4771-a06f-985a713da099-kube-api-access-kwd5z\") pod \"3e3d6a6b-d4e4-4771-a06f-985a713da099\" (UID: \"3e3d6a6b-d4e4-4771-a06f-985a713da099\") "
Jul 22 15:17:20.075815 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.075768    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e3d6a6b-d4e4-4771-a06f-985a713da099-webhook-cert\") pod \"3e3d6a6b-d4e4-4771-a06f-985a713da099\" (UID: \"3e3d6a6b-d4e4-4771-a06f-985a713da099\") "
Jul 22 15:17:20.077856 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.077822    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3d6a6b-d4e4-4771-a06f-985a713da099-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3e3d6a6b-d4e4-4771-a06f-985a713da099" (UID: "3e3d6a6b-d4e4-4771-a06f-985a713da099"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:17:20.077993 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.077893    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e3d6a6b-d4e4-4771-a06f-985a713da099-kube-api-access-kwd5z" (OuterVolumeSpecName: "kube-api-access-kwd5z") pod "3e3d6a6b-d4e4-4771-a06f-985a713da099" (UID: "3e3d6a6b-d4e4-4771-a06f-985a713da099"). InnerVolumeSpecName "kube-api-access-kwd5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:17:20.108292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.108261    2560 generic.go:356] "Generic (PLEG): container finished" podID="3e3d6a6b-d4e4-4771-a06f-985a713da099" containerID="ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc" exitCode=0
Jul 22 15:17:20.108410 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.108310    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-clm5r"
Jul 22 15:17:20.108410 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.108347    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-clm5r" event={"ID":"3e3d6a6b-d4e4-4771-a06f-985a713da099","Type":"ContainerDied","Data":"ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc"}
Jul 22 15:17:20.108410 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.108399    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-clm5r" event={"ID":"3e3d6a6b-d4e4-4771-a06f-985a713da099","Type":"ContainerDied","Data":"5fa31a008eaea9ae8c2f261773ee0d40f8652215b7230d3c85692e5e5692a7a5"}
Jul 22 15:17:20.108588 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.108425    2560 scope.go:117] "RemoveContainer" containerID="ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc"
Jul 22 15:17:20.117242 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.117225    2560 scope.go:117] "RemoveContainer" containerID="ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc"
Jul 22 15:17:20.117507 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:17:20.117489    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc\": container with ID starting with ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc not found: ID does not exist" containerID="ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc"
Jul 22 15:17:20.117562 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.117516    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc"} err="failed to get container status \"ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc\": rpc error: code = NotFound desc = could not find container \"ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc\": container with ID starting with ba08f22111b53dfc70c232bb54edbc5674a10c0e8a4e378b28e8b8aba04be2fc not found: ID does not exist"
Jul 22 15:17:20.134242 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.134217    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6995947675-clm5r"]
Jul 22 15:17:20.138858 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.138837    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6995947675-clm5r"]
Jul 22 15:17:20.177147 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.177121    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwd5z\" (UniqueName: \"kubernetes.io/projected/3e3d6a6b-d4e4-4771-a06f-985a713da099-kube-api-access-kwd5z\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:17:20.177147 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.177149    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e3d6a6b-d4e4-4771-a06f-985a713da099-webhook-cert\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:17:20.603667 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:20.603630    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e3d6a6b-d4e4-4771-a06f-985a713da099" path="/var/lib/kubelet/pods/3e3d6a6b-d4e4-4771-a06f-985a713da099/volumes"
Jul 22 15:17:21.022417 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.022382    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6995947675-wq9pq"]
Jul 22 15:17:21.022790 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.022760    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3d6a6b-d4e4-4771-a06f-985a713da099" containerName="manager"
Jul 22 15:17:21.022790 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.022773    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3d6a6b-d4e4-4771-a06f-985a713da099" containerName="manager"
Jul 22 15:17:21.022859 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.022830    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3d6a6b-d4e4-4771-a06f-985a713da099" containerName="manager"
Jul 22 15:17:21.027066 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.027048    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:17:21.029667 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.029650    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 15:17:21.030661 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.030642    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ff5sn\""
Jul 22 15:17:21.030753 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.030691    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 15:17:21.043724 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.043699    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6995947675-wq9pq"]
Jul 22 15:17:21.086821 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.086790    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/527f7621-08df-413e-9c9c-7ee256afe0e6-webhook-cert\") pod \"maas-controller-6995947675-wq9pq\" (UID: \"527f7621-08df-413e-9c9c-7ee256afe0e6\") " pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:17:21.086970 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.086837    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59v94\" (UniqueName: \"kubernetes.io/projected/527f7621-08df-413e-9c9c-7ee256afe0e6-kube-api-access-59v94\") pod \"maas-controller-6995947675-wq9pq\" (UID: \"527f7621-08df-413e-9c9c-7ee256afe0e6\") " pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:17:21.188222 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.188185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/527f7621-08df-413e-9c9c-7ee256afe0e6-webhook-cert\") pod \"maas-controller-6995947675-wq9pq\" (UID: \"527f7621-08df-413e-9c9c-7ee256afe0e6\") " pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:17:21.188419 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.188235    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59v94\" (UniqueName: \"kubernetes.io/projected/527f7621-08df-413e-9c9c-7ee256afe0e6-kube-api-access-59v94\") pod \"maas-controller-6995947675-wq9pq\" (UID: \"527f7621-08df-413e-9c9c-7ee256afe0e6\") " pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:17:21.190695 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.190667    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/527f7621-08df-413e-9c9c-7ee256afe0e6-webhook-cert\") pod \"maas-controller-6995947675-wq9pq\" (UID: \"527f7621-08df-413e-9c9c-7ee256afe0e6\") " pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:17:21.197845 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.197811    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59v94\" (UniqueName: \"kubernetes.io/projected/527f7621-08df-413e-9c9c-7ee256afe0e6-kube-api-access-59v94\") pod \"maas-controller-6995947675-wq9pq\" (UID: \"527f7621-08df-413e-9c9c-7ee256afe0e6\") " pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:17:21.337065 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.336980    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:17:21.471836 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.471813    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6995947675-wq9pq"]
Jul 22 15:17:21.474178 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:17:21.474143    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527f7621_08df_413e_9c9c_7ee256afe0e6.slice/crio-8c785ec5e46b436dd12dfbc0188f5f95da029b0653fbdde3b20fd60aea74072b WatchSource:0}: Error finding container 8c785ec5e46b436dd12dfbc0188f5f95da029b0653fbdde3b20fd60aea74072b: Status 404 returned error can't find the container with id 8c785ec5e46b436dd12dfbc0188f5f95da029b0653fbdde3b20fd60aea74072b
Jul 22 15:17:21.965238 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:21.965219    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 15:17:22.119082 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:22.119052    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-wq9pq" event={"ID":"527f7621-08df-413e-9c9c-7ee256afe0e6","Type":"ContainerStarted","Data":"96647909ba796fabe03962c19de7ae8d3405d7a38a0da61dcf0e1405383afc5d"}
Jul 22 15:17:22.119399 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:22.119110    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6995947675-wq9pq" event={"ID":"527f7621-08df-413e-9c9c-7ee256afe0e6","Type":"ContainerStarted","Data":"8c785ec5e46b436dd12dfbc0188f5f95da029b0653fbdde3b20fd60aea74072b"}
Jul 22 15:17:22.119399 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:22.119134    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:17:22.138830 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:22.138786    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6995947675-wq9pq" podStartSLOduration=0.652177208 podStartE2EDuration="1.138771357s" totalImagesPullingTime="486.594149ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:17:22.136925685 +0000 UTC m=+780.591639302" watchObservedRunningTime="2026-07-22 15:17:22.138771357 +0000 UTC m=+780.593484971"
Jul 22 15:17:33.127536 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:17:33.127502    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6995947675-wq9pq"
Jul 22 15:19:22.057842 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:19:22.057766    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:19:22.060020 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:19:22.059994    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:19:22.065082 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:19:22.065058    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:19:22.066958 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:19:22.066938    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:24:22.086696 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:24:22.086621    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:24:22.090735 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:24:22.090713    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:24:22.093110 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:24:22.093090    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:24:22.096925 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:24:22.096908    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:25:56.581344 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.581303    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"]
Jul 22 15:25:56.586908 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.586886    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.589576 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.589549    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-e0878c30-kserve-self-signed-certs\""
Jul 22 15:25:56.602281 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.602253    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"]
Jul 22 15:25:56.692374 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.692339    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-home\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.692374 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.692376    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tmp-dir\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.692620 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.692406    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfb6k\" (UniqueName: \"kubernetes.io/projected/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-kube-api-access-nfb6k\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.692620 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.692489    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-dshm\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.692620 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.692537    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tls-certs\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.692754 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.692628    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-model-cache\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.793882 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.793846    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-home\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.793882 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.793887    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tmp-dir\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.794126 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.794004    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfb6k\" (UniqueName: \"kubernetes.io/projected/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-kube-api-access-nfb6k\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.794126 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.794072    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-dshm\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.794126 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.794116    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tls-certs\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.794287 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.794169    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-model-cache\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.794287 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.794215    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tmp-dir\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.794287 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.794272    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-home\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.794476 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.794444    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-model-cache\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.796147 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.796126    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-dshm\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.796542 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.796522    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tls-certs\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.802348 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.802319    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfb6k\" (UniqueName: \"kubernetes.io/projected/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-kube-api-access-nfb6k\") pod \"e2e-same-id-a-e0878c30-kserve-554f445d55-88c28\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") " pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:56.895701 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:56.895673    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:25:57.030993 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:57.030967    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"]
Jul 22 15:25:57.032957 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:25:57.032924    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e13b17d_89fd_4a50_9d7a_b3bedd5e53f1.slice/crio-62e4d5f6600e5899db05ff378f5552700ca8ce0a671af2341246fbc527a0b310 WatchSource:0}: Error finding container 62e4d5f6600e5899db05ff378f5552700ca8ce0a671af2341246fbc527a0b310: Status 404 returned error can't find the container with id 62e4d5f6600e5899db05ff378f5552700ca8ce0a671af2341246fbc527a0b310
Jul 22 15:25:57.467314 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:57.467273    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28" event={"ID":"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1","Type":"ContainerStarted","Data":"33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6"}
Jul 22 15:25:57.467314 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:57.467320    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28" event={"ID":"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1","Type":"ContainerStarted","Data":"62e4d5f6600e5899db05ff378f5552700ca8ce0a671af2341246fbc527a0b310"}
Jul 22 15:25:57.487377 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:57.487318    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28" podStartSLOduration=1.487303066 podStartE2EDuration="1.487303066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:25:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:25:57.485855412 +0000 UTC m=+1295.940569025" watchObservedRunningTime="2026-07-22 15:25:57.487303066 +0000 UTC m=+1295.942016682"
Jul 22 15:25:58.768011 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.767971    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"]
Jul 22 15:25:58.771439 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.771422    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:58.774292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.774270    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e0878c30-kserve-self-signed-certs\""
Jul 22 15:25:58.784026 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.784002    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"]
Jul 22 15:25:58.914900 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.914862    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghqr6\" (UniqueName: \"kubernetes.io/projected/19f331eb-fba9-47fe-bd53-07d1fe02033b-kube-api-access-ghqr6\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:58.915089 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.914928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-dshm\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:58.915089 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.914987    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-tmp-dir\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:58.915089 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.915028    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-home\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:58.915089 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.915052    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-model-cache\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:58.915533 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:58.915130    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f331eb-fba9-47fe-bd53-07d1fe02033b-tls-certs\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.016331 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.016294    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-home\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.016331 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.016332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-model-cache\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.016598 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.016382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f331eb-fba9-47fe-bd53-07d1fe02033b-tls-certs\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.016598 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.016434    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghqr6\" (UniqueName: \"kubernetes.io/projected/19f331eb-fba9-47fe-bd53-07d1fe02033b-kube-api-access-ghqr6\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.016598 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.016501    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-dshm\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.016598 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.016519    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-tmp-dir\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.016811 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.016773    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-home\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.016856 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.016806    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-model-cache\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.016895 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.016862    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-tmp-dir\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.018789 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.018734    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-dshm\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.019134 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.019114    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f331eb-fba9-47fe-bd53-07d1fe02033b-tls-certs\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.024206 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.024180    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghqr6\" (UniqueName: \"kubernetes.io/projected/19f331eb-fba9-47fe-bd53-07d1fe02033b-kube-api-access-ghqr6\") pod \"e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") " pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.081525 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.081491    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:25:59.215370 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.215337    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"]
Jul 22 15:25:59.477778 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.477744    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk" event={"ID":"19f331eb-fba9-47fe-bd53-07d1fe02033b","Type":"ContainerStarted","Data":"160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df"}
Jul 22 15:25:59.477972 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.477788    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk" event={"ID":"19f331eb-fba9-47fe-bd53-07d1fe02033b","Type":"ContainerStarted","Data":"b65132221fc571711c4a6fc32b97465dbd9902a85969ccdd4be5f78a9136d111"}
Jul 22 15:25:59.495238 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:25:59.495185    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk" podStartSLOduration=1.495167884 podStartE2EDuration="1.495167884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:25:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:25:59.493733087 +0000 UTC m=+1297.948446703" watchObservedRunningTime="2026-07-22 15:25:59.495167884 +0000 UTC m=+1297.949881549"
Jul 22 15:26:06.895840 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:06.895798    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:26:06.896216 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:06.895857    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:26:06.908535 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:06.908505    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:26:07.530026 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:07.529996    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:26:09.082258 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:09.082217    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:26:09.082747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:09.082272    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:26:09.094807 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:09.094778    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:26:09.542990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:09.542960    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:26:20.068989 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:20.068942    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"]
Jul 22 15:26:21.555039 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:21.554992    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"]
Jul 22 15:26:35.089163 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.089101    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28" podUID="0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" containerName="main" containerID="cri-o://33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6" gracePeriod=45
Jul 22 15:26:35.332759 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.332735    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:26:35.462224 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.462190    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tls-certs\") pod \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") "
Jul 22 15:26:35.462411 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.462240    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfb6k\" (UniqueName: \"kubernetes.io/projected/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-kube-api-access-nfb6k\") pod \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") "
Jul 22 15:26:35.462411 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.462265    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-model-cache\") pod \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") "
Jul 22 15:26:35.462411 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.462291    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-home\") pod \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") "
Jul 22 15:26:35.462411 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.462320    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tmp-dir\") pod \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") "
Jul 22 15:26:35.462411 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.462377    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-dshm\") pod \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\" (UID: \"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1\") "
Jul 22 15:26:35.462690 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.462605    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" (UID: "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:26:35.462923 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.462611    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-home" (OuterVolumeSpecName: "home") pod "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" (UID: "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:26:35.463149 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.462676    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" (UID: "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:26:35.469930 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.465580    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" (UID: "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:26:35.469930 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.465672    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-kube-api-access-nfb6k" (OuterVolumeSpecName: "kube-api-access-nfb6k") pod "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" (UID: "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1"). InnerVolumeSpecName "kube-api-access-nfb6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:26:35.470146 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.470121    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-dshm" (OuterVolumeSpecName: "dshm") pod "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" (UID: "0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:26:35.564230 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.564188    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:35.564230 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.564221    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:35.564462 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.564263    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfb6k\" (UniqueName: \"kubernetes.io/projected/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-kube-api-access-nfb6k\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:35.564462 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.564278    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:35.564462 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.564289    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:35.564462 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.564299    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:35.647337 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.647301    2560 generic.go:356] "Generic (PLEG): container finished" podID="0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" containerID="33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6" exitCode=0
Jul 22 15:26:35.647534 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.647359    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"
Jul 22 15:26:35.647534 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.647389    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28" event={"ID":"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1","Type":"ContainerDied","Data":"33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6"}
Jul 22 15:26:35.647534 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.647439    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28" event={"ID":"0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1","Type":"ContainerDied","Data":"62e4d5f6600e5899db05ff378f5552700ca8ce0a671af2341246fbc527a0b310"}
Jul 22 15:26:35.647534 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.647489    2560 scope.go:117] "RemoveContainer" containerID="33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6"
Jul 22 15:26:35.656799 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.656779    2560 scope.go:117] "RemoveContainer" containerID="33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6"
Jul 22 15:26:35.657031 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:26:35.657012    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6\": container with ID starting with 33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6 not found: ID does not exist" containerID="33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6"
Jul 22 15:26:35.657082 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.657039    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6"} err="failed to get container status \"33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6\": rpc error: code = NotFound desc = could not find container \"33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6\": container with ID starting with 33a1d72670298c9337290875a27a9f8c499a7690c919d057b20bec52613067f6 not found: ID does not exist"
Jul 22 15:26:35.668880 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.668846    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"]
Jul 22 15:26:35.670619 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:35.670595    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-e0878c30-kserve-554f445d55-88c28"]
Jul 22 15:26:36.577619 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.577575    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk" podUID="19f331eb-fba9-47fe-bd53-07d1fe02033b" containerName="main" containerID="cri-o://160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df" gracePeriod=45
Jul 22 15:26:36.603844 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.603810    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" path="/var/lib/kubelet/pods/0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1/volumes"
Jul 22 15:26:36.835953 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.835890    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:26:36.976023 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.975982    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghqr6\" (UniqueName: \"kubernetes.io/projected/19f331eb-fba9-47fe-bd53-07d1fe02033b-kube-api-access-ghqr6\") pod \"19f331eb-fba9-47fe-bd53-07d1fe02033b\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") "
Jul 22 15:26:36.976225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.976035    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-dshm\") pod \"19f331eb-fba9-47fe-bd53-07d1fe02033b\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") "
Jul 22 15:26:36.976225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.976059    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f331eb-fba9-47fe-bd53-07d1fe02033b-tls-certs\") pod \"19f331eb-fba9-47fe-bd53-07d1fe02033b\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") "
Jul 22 15:26:36.976225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.976088    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-tmp-dir\") pod \"19f331eb-fba9-47fe-bd53-07d1fe02033b\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") "
Jul 22 15:26:36.976225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.976178    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-model-cache\") pod \"19f331eb-fba9-47fe-bd53-07d1fe02033b\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") "
Jul 22 15:26:36.976225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.976208    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-home\") pod \"19f331eb-fba9-47fe-bd53-07d1fe02033b\" (UID: \"19f331eb-fba9-47fe-bd53-07d1fe02033b\") "
Jul 22 15:26:36.976521 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.976488    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19f331eb-fba9-47fe-bd53-07d1fe02033b" (UID: "19f331eb-fba9-47fe-bd53-07d1fe02033b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:26:36.976521 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.976503    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-model-cache" (OuterVolumeSpecName: "model-cache") pod "19f331eb-fba9-47fe-bd53-07d1fe02033b" (UID: "19f331eb-fba9-47fe-bd53-07d1fe02033b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:26:36.976619 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.976547    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-home" (OuterVolumeSpecName: "home") pod "19f331eb-fba9-47fe-bd53-07d1fe02033b" (UID: "19f331eb-fba9-47fe-bd53-07d1fe02033b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:26:36.978265 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.978243    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-dshm" (OuterVolumeSpecName: "dshm") pod "19f331eb-fba9-47fe-bd53-07d1fe02033b" (UID: "19f331eb-fba9-47fe-bd53-07d1fe02033b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:26:36.978703 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.978683    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19f331eb-fba9-47fe-bd53-07d1fe02033b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19f331eb-fba9-47fe-bd53-07d1fe02033b" (UID: "19f331eb-fba9-47fe-bd53-07d1fe02033b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:26:36.978783 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:36.978705    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19f331eb-fba9-47fe-bd53-07d1fe02033b-kube-api-access-ghqr6" (OuterVolumeSpecName: "kube-api-access-ghqr6") pod "19f331eb-fba9-47fe-bd53-07d1fe02033b" (UID: "19f331eb-fba9-47fe-bd53-07d1fe02033b"). InnerVolumeSpecName "kube-api-access-ghqr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:26:37.077798 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.077757    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:37.077798 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.077794    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:37.077798 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.077807    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghqr6\" (UniqueName: \"kubernetes.io/projected/19f331eb-fba9-47fe-bd53-07d1fe02033b-kube-api-access-ghqr6\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:37.078061 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.077819    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:37.078061 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.077838    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f331eb-fba9-47fe-bd53-07d1fe02033b-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:37.078061 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.077850    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f331eb-fba9-47fe-bd53-07d1fe02033b-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:26:37.659738 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.659696    2560 generic.go:356] "Generic (PLEG): container finished" podID="19f331eb-fba9-47fe-bd53-07d1fe02033b" containerID="160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df" exitCode=0
Jul 22 15:26:37.660181 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.659745    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"
Jul 22 15:26:37.660181 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.659781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk" event={"ID":"19f331eb-fba9-47fe-bd53-07d1fe02033b","Type":"ContainerDied","Data":"160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df"}
Jul 22 15:26:37.660181 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.659826    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk" event={"ID":"19f331eb-fba9-47fe-bd53-07d1fe02033b","Type":"ContainerDied","Data":"b65132221fc571711c4a6fc32b97465dbd9902a85969ccdd4be5f78a9136d111"}
Jul 22 15:26:37.660181 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.659845    2560 scope.go:117] "RemoveContainer" containerID="160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df"
Jul 22 15:26:37.668791 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.668769    2560 scope.go:117] "RemoveContainer" containerID="160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df"
Jul 22 15:26:37.669085 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:26:37.669062    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df\": container with ID starting with 160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df not found: ID does not exist" containerID="160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df"
Jul 22 15:26:37.669124 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.669095    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df"} err="failed to get container status \"160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df\": rpc error: code = NotFound desc = could not find container \"160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df\": container with ID starting with 160ede75b6cbc10fad0d87cffbc620fe755daa2d556a3506679c4512d4a277df not found: ID does not exist"
Jul 22 15:26:37.681043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.681016    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"]
Jul 22 15:26:37.685125 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:37.685100    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e0878c30-kserve-585c6fd799-swppk"]
Jul 22 15:26:38.602542 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:26:38.602509    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19f331eb-fba9-47fe-bd53-07d1fe02033b" path="/var/lib/kubelet/pods/19f331eb-fba9-47fe-bd53-07d1fe02033b/volumes"
Jul 22 15:29:22.115667 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:29:22.115635    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:29:22.124631 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:29:22.124603    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:29:22.125900 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:29:22.125880    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:29:22.131695 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:29:22.131673    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:30:00.133995 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.133909    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm"]
Jul 22 15:30:00.134616 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.134559    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" containerName="main"
Jul 22 15:30:00.134616 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.134582    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" containerName="main"
Jul 22 15:30:00.134616 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.134595    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f331eb-fba9-47fe-bd53-07d1fe02033b" containerName="main"
Jul 22 15:30:00.134616 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.134604    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f331eb-fba9-47fe-bd53-07d1fe02033b" containerName="main"
Jul 22 15:30:00.134833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.134699    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e13b17d-89fd-4a50-9d7a-b3bedd5e53f1" containerName="main"
Jul 22 15:30:00.134833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.134715    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="19f331eb-fba9-47fe-bd53-07d1fe02033b" containerName="main"
Jul 22 15:30:00.138054 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.138029    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm"
Jul 22 15:30:00.141209 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.141187    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:30:00.147491 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.147410    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm"]
Jul 22 15:30:00.149561 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.149543    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm"
Jul 22 15:30:00.288539 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.288514    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm"]
Jul 22 15:30:00.291256 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:30:00.291224    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25f7afb1_8597_4d0d_8ba7_e0c923a5194f.slice/crio-2ad4a9e9b767f80aa2cf987dd1c84db3610510508a38337c6b2bc4de92969c8b WatchSource:0}: Error finding container 2ad4a9e9b767f80aa2cf987dd1c84db3610510508a38337c6b2bc4de92969c8b: Status 404 returned error can't find the container with id 2ad4a9e9b767f80aa2cf987dd1c84db3610510508a38337c6b2bc4de92969c8b
Jul 22 15:30:00.293191 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.293171    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:30:00.608195 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:00.608157    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm" event={"ID":"25f7afb1-8597-4d0d-8ba7-e0c923a5194f","Type":"ContainerStarted","Data":"2ad4a9e9b767f80aa2cf987dd1c84db3610510508a38337c6b2bc4de92969c8b"}
Jul 22 15:30:03.623269 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:03.623178    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm" event={"ID":"25f7afb1-8597-4d0d-8ba7-e0c923a5194f","Type":"ContainerStarted","Data":"775f5cf348efa601fab8ca06d4e2bc494e7cc5f9f17e21583e369eaaa7c42cd6"}
Jul 22 15:30:03.638087 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:03.638028    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm" podStartSLOduration=0.834178484 podStartE2EDuration="3.63801187s" totalImagesPullingTime="2.803833386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:30:03.637156797 +0000 UTC m=+1542.091870412" watchObservedRunningTime="2026-07-22 15:30:03.63801187 +0000 UTC m=+1542.092725484"
Jul 22 15:30:04.628460 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:04.628412    2560 generic.go:356] "Generic (PLEG): container finished" podID="25f7afb1-8597-4d0d-8ba7-e0c923a5194f" containerID="775f5cf348efa601fab8ca06d4e2bc494e7cc5f9f17e21583e369eaaa7c42cd6" exitCode=0
Jul 22 15:30:04.628841 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:04.628503    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm" event={"ID":"25f7afb1-8597-4d0d-8ba7-e0c923a5194f","Type":"ContainerDied","Data":"775f5cf348efa601fab8ca06d4e2bc494e7cc5f9f17e21583e369eaaa7c42cd6"}
Jul 22 15:30:05.772304 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:05.772274    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm"
Jul 22 15:30:06.638623 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:06.638590    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm"
Jul 22 15:30:06.638623 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:06.638602    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745570-cpmcm" event={"ID":"25f7afb1-8597-4d0d-8ba7-e0c923a5194f","Type":"ContainerDied","Data":"2ad4a9e9b767f80aa2cf987dd1c84db3610510508a38337c6b2bc4de92969c8b"}
Jul 22 15:30:06.638875 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:06.638645    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ad4a9e9b767f80aa2cf987dd1c84db3610510508a38337c6b2bc4de92969c8b"
Jul 22 15:30:27.558263 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.558224    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"]
Jul 22 15:30:27.558694 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.558669    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25f7afb1-8597-4d0d-8ba7-e0c923a5194f" containerName="cleanup"
Jul 22 15:30:27.558694 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.558683    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="25f7afb1-8597-4d0d-8ba7-e0c923a5194f" containerName="cleanup"
Jul 22 15:30:27.558780 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.558748    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="25f7afb1-8597-4d0d-8ba7-e0c923a5194f" containerName="cleanup"
Jul 22 15:30:27.561864 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.561843    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"
Jul 22 15:30:27.564555 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.564540    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:30:27.570439 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.570412    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"]
Jul 22 15:30:27.592600 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.592566    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1de5279-35b7-42a5-a493-421378cb2b20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf\" (UID: \"b1de5279-35b7-42a5-a493-421378cb2b20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"
Jul 22 15:30:27.693054 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.693020    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1de5279-35b7-42a5-a493-421378cb2b20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf\" (UID: \"b1de5279-35b7-42a5-a493-421378cb2b20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"
Jul 22 15:30:27.693708 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.693690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1de5279-35b7-42a5-a493-421378cb2b20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf\" (UID: \"b1de5279-35b7-42a5-a493-421378cb2b20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"
Jul 22 15:30:27.873876 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:27.873774    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"
Jul 22 15:30:28.007893 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:28.007861    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"]
Jul 22 15:30:28.011816 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:30:28.011790    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1de5279_35b7_42a5_a493_421378cb2b20.slice/crio-17bf4fd7fb30ce85b3583a22e6bcf1815478ddcffb7f54abc80931631449a98e WatchSource:0}: Error finding container 17bf4fd7fb30ce85b3583a22e6bcf1815478ddcffb7f54abc80931631449a98e: Status 404 returned error can't find the container with id 17bf4fd7fb30ce85b3583a22e6bcf1815478ddcffb7f54abc80931631449a98e
Jul 22 15:30:28.739805 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:28.739774    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf" event={"ID":"b1de5279-35b7-42a5-a493-421378cb2b20","Type":"ContainerStarted","Data":"fc6ec3744c3cd332cca86cd66a83ef0b815e78faffb2a7bc1460e7a8893c0d3c"}
Jul 22 15:30:28.740107 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:28.739840    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf" event={"ID":"b1de5279-35b7-42a5-a493-421378cb2b20","Type":"ContainerStarted","Data":"17bf4fd7fb30ce85b3583a22e6bcf1815478ddcffb7f54abc80931631449a98e"}
Jul 22 15:30:29.745767 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:29.745728    2560 generic.go:356] "Generic (PLEG): container finished" podID="b1de5279-35b7-42a5-a493-421378cb2b20" containerID="fc6ec3744c3cd332cca86cd66a83ef0b815e78faffb2a7bc1460e7a8893c0d3c" exitCode=0
Jul 22 15:30:29.746190 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:29.745809    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf" event={"ID":"b1de5279-35b7-42a5-a493-421378cb2b20","Type":"ContainerDied","Data":"fc6ec3744c3cd332cca86cd66a83ef0b815e78faffb2a7bc1460e7a8893c0d3c"}
Jul 22 15:30:29.881556 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:29.881528    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"
Jul 22 15:30:29.912756 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:29.912725    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1de5279-35b7-42a5-a493-421378cb2b20-maas-api-service-ca\") pod \"b1de5279-35b7-42a5-a493-421378cb2b20\" (UID: \"b1de5279-35b7-42a5-a493-421378cb2b20\") "
Jul 22 15:30:29.913199 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:29.913170    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1de5279-35b7-42a5-a493-421378cb2b20-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b1de5279-35b7-42a5-a493-421378cb2b20" (UID: "b1de5279-35b7-42a5-a493-421378cb2b20"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:30:30.013642 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:30.013549    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1de5279-35b7-42a5-a493-421378cb2b20-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:30:30.751558 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:30.751534    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"
Jul 22 15:30:30.751907 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:30.751554    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf" event={"ID":"b1de5279-35b7-42a5-a493-421378cb2b20","Type":"ContainerDied","Data":"17bf4fd7fb30ce85b3583a22e6bcf1815478ddcffb7f54abc80931631449a98e"}
Jul 22 15:30:30.751907 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:30.751602    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17bf4fd7fb30ce85b3583a22e6bcf1815478ddcffb7f54abc80931631449a98e"
Jul 22 15:30:42.638048 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:42.638011    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"]
Jul 22 15:30:42.640001 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:42.639973    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c6b1b943-nl6mf"]
Jul 22 15:30:44.602130 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:30:44.602097    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1de5279-35b7-42a5-a493-421378cb2b20" path="/var/lib/kubelet/pods/b1de5279-35b7-42a5-a493-421378cb2b20/volumes"
Jul 22 15:31:11.248742 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.248708    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"]
Jul 22 15:31:11.249088 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.249064    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1de5279-35b7-42a5-a493-421378cb2b20" containerName="revoke-keys"
Jul 22 15:31:11.249088 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.249075    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1de5279-35b7-42a5-a493-421378cb2b20" containerName="revoke-keys"
Jul 22 15:31:11.249161 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.249145    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1de5279-35b7-42a5-a493-421378cb2b20" containerName="revoke-keys"
Jul 22 15:31:11.252154 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.252137    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"
Jul 22 15:31:11.255588 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.255564    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:31:11.261943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.261919    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"]
Jul 22 15:31:11.381513 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.381475    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d477394-49bb-4e5d-b504-f17779a506f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7148173f-f8844\" (UID: \"0d477394-49bb-4e5d-b504-f17779a506f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"
Jul 22 15:31:11.481999 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.481963    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d477394-49bb-4e5d-b504-f17779a506f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7148173f-f8844\" (UID: \"0d477394-49bb-4e5d-b504-f17779a506f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"
Jul 22 15:31:11.482751 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.482730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d477394-49bb-4e5d-b504-f17779a506f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7148173f-f8844\" (UID: \"0d477394-49bb-4e5d-b504-f17779a506f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"
Jul 22 15:31:11.563477 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.563376    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"
Jul 22 15:31:11.693169 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.693137    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"]
Jul 22 15:31:11.697312 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:31:11.697281    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d477394_49bb_4e5d_b504_f17779a506f7.slice/crio-d114c7ac5d0417a3cf1a855404eb75ece2b62594a4aab56a3fc9887f4ad09b75 WatchSource:0}: Error finding container d114c7ac5d0417a3cf1a855404eb75ece2b62594a4aab56a3fc9887f4ad09b75: Status 404 returned error can't find the container with id d114c7ac5d0417a3cf1a855404eb75ece2b62594a4aab56a3fc9887f4ad09b75
Jul 22 15:31:11.938513 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:11.938480    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844" event={"ID":"0d477394-49bb-4e5d-b504-f17779a506f7","Type":"ContainerStarted","Data":"d114c7ac5d0417a3cf1a855404eb75ece2b62594a4aab56a3fc9887f4ad09b75"}
Jul 22 15:31:12.944949 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:12.944918    2560 generic.go:356] "Generic (PLEG): container finished" podID="0d477394-49bb-4e5d-b504-f17779a506f7" containerID="a775024932c77f9f59b34cbe505294cc175065e0c9169a2a92ce1aff7ec3ad5d" exitCode=0
Jul 22 15:31:12.945307 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:12.945004    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844" event={"ID":"0d477394-49bb-4e5d-b504-f17779a506f7","Type":"ContainerDied","Data":"a775024932c77f9f59b34cbe505294cc175065e0c9169a2a92ce1aff7ec3ad5d"}
Jul 22 15:31:14.080467 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:14.080427    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"
Jul 22 15:31:14.106196 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:14.106169    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d477394-49bb-4e5d-b504-f17779a506f7-maas-api-service-ca\") pod \"0d477394-49bb-4e5d-b504-f17779a506f7\" (UID: \"0d477394-49bb-4e5d-b504-f17779a506f7\") "
Jul 22 15:31:14.106631 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:14.106604    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d477394-49bb-4e5d-b504-f17779a506f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0d477394-49bb-4e5d-b504-f17779a506f7" (UID: "0d477394-49bb-4e5d-b504-f17779a506f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:31:14.207189 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:14.207101    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d477394-49bb-4e5d-b504-f17779a506f7-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:31:14.955602 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:14.955574    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"
Jul 22 15:31:14.955602 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:14.955600    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844" event={"ID":"0d477394-49bb-4e5d-b504-f17779a506f7","Type":"ContainerDied","Data":"d114c7ac5d0417a3cf1a855404eb75ece2b62594a4aab56a3fc9887f4ad09b75"}
Jul 22 15:31:14.955823 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:14.955643    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d114c7ac5d0417a3cf1a855404eb75ece2b62594a4aab56a3fc9887f4ad09b75"
Jul 22 15:31:26.329716 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:26.329655    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"]
Jul 22 15:31:26.331746 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:26.331719    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7148173f-f8844"]
Jul 22 15:31:26.603009 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:26.602926    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d477394-49bb-4e5d-b504-f17779a506f7" path="/var/lib/kubelet/pods/0d477394-49bb-4e5d-b504-f17779a506f7/volumes"
Jul 22 15:31:29.387224 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.387187    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"]
Jul 22 15:31:29.387900 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.387879    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d477394-49bb-4e5d-b504-f17779a506f7" containerName="revoke-keys"
Jul 22 15:31:29.387951 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.387905    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d477394-49bb-4e5d-b504-f17779a506f7" containerName="revoke-keys"
Jul 22 15:31:29.388061 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.388048    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d477394-49bb-4e5d-b504-f17779a506f7" containerName="revoke-keys"
Jul 22 15:31:29.391666 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.391643    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:29.394524 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.394504    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-17acd718-serving-cert\""
Jul 22 15:31:29.394689 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.394598    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8q48q\""
Jul 22 15:31:29.402822 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.402800    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"]
Jul 22 15:31:29.446442 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.446405    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf9368b-5240-4490-a732-fa57cea69303-maas-api-tls\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:29.446624 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.446505    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf9368b-5240-4490-a732-fa57cea69303-openshift-service-ca\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:29.446624 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.446532    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hks5r\" (UniqueName: \"kubernetes.io/projected/caf9368b-5240-4490-a732-fa57cea69303-kube-api-access-hks5r\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:29.547188 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.547145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf9368b-5240-4490-a732-fa57cea69303-openshift-service-ca\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:29.547188 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.547184    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hks5r\" (UniqueName: \"kubernetes.io/projected/caf9368b-5240-4490-a732-fa57cea69303-kube-api-access-hks5r\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:29.547400 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.547255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf9368b-5240-4490-a732-fa57cea69303-maas-api-tls\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:29.547400 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:31:29.547374    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-serving-cert: secret "maas-api-e2e-ait-17acd718-serving-cert" not found
Jul 22 15:31:29.547503 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:31:29.547496    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caf9368b-5240-4490-a732-fa57cea69303-maas-api-tls podName:caf9368b-5240-4490-a732-fa57cea69303 nodeName:}" failed. No retries permitted until 2026-07-22 15:31:30.047431189 +0000 UTC m=+1628.502144799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/caf9368b-5240-4490-a732-fa57cea69303-maas-api-tls") pod "maas-api-e2e-ait-17acd718-55b65d9675-m6mwm" (UID: "caf9368b-5240-4490-a732-fa57cea69303") : secret "maas-api-e2e-ait-17acd718-serving-cert" not found
Jul 22 15:31:29.547889 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.547869    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf9368b-5240-4490-a732-fa57cea69303-openshift-service-ca\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:29.556049 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:29.556015    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hks5r\" (UniqueName: \"kubernetes.io/projected/caf9368b-5240-4490-a732-fa57cea69303-kube-api-access-hks5r\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:30.052997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:30.052952    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf9368b-5240-4490-a732-fa57cea69303-maas-api-tls\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:30.055472 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:30.055413    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf9368b-5240-4490-a732-fa57cea69303-maas-api-tls\") pod \"maas-api-e2e-ait-17acd718-55b65d9675-m6mwm\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:30.302787 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:30.302749    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:30.449268 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:30.449242    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"]
Jul 22 15:31:30.451675 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:31:30.451645    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf9368b_5240_4490_a732_fa57cea69303.slice/crio-6b2809a9a51b13728ce8deb1e9e7cd29ad6cab1448bc78be6903226aa5d26d9d WatchSource:0}: Error finding container 6b2809a9a51b13728ce8deb1e9e7cd29ad6cab1448bc78be6903226aa5d26d9d: Status 404 returned error can't find the container with id 6b2809a9a51b13728ce8deb1e9e7cd29ad6cab1448bc78be6903226aa5d26d9d
Jul 22 15:31:31.037753 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:31.037718    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm" event={"ID":"caf9368b-5240-4490-a732-fa57cea69303","Type":"ContainerStarted","Data":"6b2809a9a51b13728ce8deb1e9e7cd29ad6cab1448bc78be6903226aa5d26d9d"}
Jul 22 15:31:33.049013 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:33.048980    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm" event={"ID":"caf9368b-5240-4490-a732-fa57cea69303","Type":"ContainerStarted","Data":"b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94"}
Jul 22 15:31:33.049427 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:33.049236    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:31:33.066738 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:33.066690    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm" podStartSLOduration=1.99646966 podStartE2EDuration="4.066677591s" totalImagesPullingTime="2.070207931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:31:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:31:33.065902321 +0000 UTC m=+1631.520615936" watchObservedRunningTime="2026-07-22 15:31:33.066677591 +0000 UTC m=+1631.521391206"
Jul 22 15:31:39.060650 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:31:39.060618    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:32:05.703410 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:05.703375    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"]
Jul 22 15:32:05.703899 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:05.703615    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm" podUID="caf9368b-5240-4490-a732-fa57cea69303" containerName="maas-api" containerID="cri-o://b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94" gracePeriod=30
Jul 22 15:32:05.947514 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:05.947489    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:32:05.993263 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:05.993173    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf9368b-5240-4490-a732-fa57cea69303-maas-api-tls\") pod \"caf9368b-5240-4490-a732-fa57cea69303\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") "
Jul 22 15:32:05.993443 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:05.993274    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf9368b-5240-4490-a732-fa57cea69303-openshift-service-ca\") pod \"caf9368b-5240-4490-a732-fa57cea69303\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") "
Jul 22 15:32:05.993443 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:05.993310    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hks5r\" (UniqueName: \"kubernetes.io/projected/caf9368b-5240-4490-a732-fa57cea69303-kube-api-access-hks5r\") pod \"caf9368b-5240-4490-a732-fa57cea69303\" (UID: \"caf9368b-5240-4490-a732-fa57cea69303\") "
Jul 22 15:32:05.993756 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:05.993727    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf9368b-5240-4490-a732-fa57cea69303-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "caf9368b-5240-4490-a732-fa57cea69303" (UID: "caf9368b-5240-4490-a732-fa57cea69303"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:32:05.995659 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:05.995626    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf9368b-5240-4490-a732-fa57cea69303-kube-api-access-hks5r" (OuterVolumeSpecName: "kube-api-access-hks5r") pod "caf9368b-5240-4490-a732-fa57cea69303" (UID: "caf9368b-5240-4490-a732-fa57cea69303"). InnerVolumeSpecName "kube-api-access-hks5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:32:05.995799 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:05.995672    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf9368b-5240-4490-a732-fa57cea69303-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "caf9368b-5240-4490-a732-fa57cea69303" (UID: "caf9368b-5240-4490-a732-fa57cea69303"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:32:06.093977 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.093940    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caf9368b-5240-4490-a732-fa57cea69303-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:32:06.093977 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.093972    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caf9368b-5240-4490-a732-fa57cea69303-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:32:06.093977 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.093986    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hks5r\" (UniqueName: \"kubernetes.io/projected/caf9368b-5240-4490-a732-fa57cea69303-kube-api-access-hks5r\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:32:06.206392 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.206350    2560 generic.go:356] "Generic (PLEG): container finished" podID="caf9368b-5240-4490-a732-fa57cea69303" containerID="b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94" exitCode=0
Jul 22 15:32:06.206591 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.206402    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"
Jul 22 15:32:06.206591 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.206434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm" event={"ID":"caf9368b-5240-4490-a732-fa57cea69303","Type":"ContainerDied","Data":"b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94"}
Jul 22 15:32:06.206591 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.206533    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm" event={"ID":"caf9368b-5240-4490-a732-fa57cea69303","Type":"ContainerDied","Data":"6b2809a9a51b13728ce8deb1e9e7cd29ad6cab1448bc78be6903226aa5d26d9d"}
Jul 22 15:32:06.206591 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.206557    2560 scope.go:117] "RemoveContainer" containerID="b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94"
Jul 22 15:32:06.215329 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.215314    2560 scope.go:117] "RemoveContainer" containerID="b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94"
Jul 22 15:32:06.215590 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:32:06.215571    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94\": container with ID starting with b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94 not found: ID does not exist" containerID="b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94"
Jul 22 15:32:06.215666 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.215597    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94"} err="failed to get container status \"b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94\": rpc error: code = NotFound desc = could not find container \"b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94\": container with ID starting with b7bc2f8995004a37539cd4dde2d5ae23a69bf0969197ad7260051ced35dfcc94 not found: ID does not exist"
Jul 22 15:32:06.228154 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.228132    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"]
Jul 22 15:32:06.234589 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.234564    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17acd718-55b65d9675-m6mwm"]
Jul 22 15:32:06.602054 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:06.602023    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caf9368b-5240-4490-a732-fa57cea69303" path="/var/lib/kubelet/pods/caf9368b-5240-4490-a732-fa57cea69303/volumes"
Jul 22 15:32:14.616852 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.616813    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"]
Jul 22 15:32:14.617306 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.617193    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf9368b-5240-4490-a732-fa57cea69303" containerName="maas-api"
Jul 22 15:32:14.617306 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.617204    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf9368b-5240-4490-a732-fa57cea69303" containerName="maas-api"
Jul 22 15:32:14.617306 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.617269    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf9368b-5240-4490-a732-fa57cea69303" containerName="maas-api"
Jul 22 15:32:14.621708 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.621690    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:14.624918 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.624897    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-5fe701b2-serving-cert\""
Jul 22 15:32:14.625034 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.624942    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8q48q\""
Jul 22 15:32:14.631020 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.630947    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"]
Jul 22 15:32:14.675638 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.675589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpbmk\" (UniqueName: \"kubernetes.io/projected/99ed7f98-651b-41d0-a915-7a501ac6f29a-kube-api-access-vpbmk\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:14.675813 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.675728    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ed7f98-651b-41d0-a915-7a501ac6f29a-openshift-service-ca\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:14.675813 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.675781    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99ed7f98-651b-41d0-a915-7a501ac6f29a-maas-api-tls\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:14.776579 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.776538    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ed7f98-651b-41d0-a915-7a501ac6f29a-openshift-service-ca\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:14.776750 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.776606    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99ed7f98-651b-41d0-a915-7a501ac6f29a-maas-api-tls\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:14.776750 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.776670    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpbmk\" (UniqueName: \"kubernetes.io/projected/99ed7f98-651b-41d0-a915-7a501ac6f29a-kube-api-access-vpbmk\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:14.776837 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:32:14.776762    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-serving-cert: secret "maas-api-e2e-derive-5fe701b2-serving-cert" not found
Jul 22 15:32:14.776932 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:32:14.776844    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99ed7f98-651b-41d0-a915-7a501ac6f29a-maas-api-tls podName:99ed7f98-651b-41d0-a915-7a501ac6f29a nodeName:}" failed. No retries permitted until 2026-07-22 15:32:15.276826016 +0000 UTC m=+1673.731539612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99ed7f98-651b-41d0-a915-7a501ac6f29a-maas-api-tls") pod "maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch" (UID: "99ed7f98-651b-41d0-a915-7a501ac6f29a") : secret "maas-api-e2e-derive-5fe701b2-serving-cert" not found
Jul 22 15:32:14.777292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.777272    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ed7f98-651b-41d0-a915-7a501ac6f29a-openshift-service-ca\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:14.791876 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:14.791845    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpbmk\" (UniqueName: \"kubernetes.io/projected/99ed7f98-651b-41d0-a915-7a501ac6f29a-kube-api-access-vpbmk\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:15.281619 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:15.281585    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99ed7f98-651b-41d0-a915-7a501ac6f29a-maas-api-tls\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:15.284224 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:15.284199    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99ed7f98-651b-41d0-a915-7a501ac6f29a-maas-api-tls\") pod \"maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:15.533555 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:15.533464    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:15.670731 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:15.670695    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"]
Jul 22 15:32:15.675240 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:32:15.675213    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ed7f98_651b_41d0_a915_7a501ac6f29a.slice/crio-5729e0476e8af8dfcd8207c3826a558fdc6698d5651aaf6fd898cf5d80c970b6 WatchSource:0}: Error finding container 5729e0476e8af8dfcd8207c3826a558fdc6698d5651aaf6fd898cf5d80c970b6: Status 404 returned error can't find the container with id 5729e0476e8af8dfcd8207c3826a558fdc6698d5651aaf6fd898cf5d80c970b6
Jul 22 15:32:16.264709 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:16.264671    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch" event={"ID":"99ed7f98-651b-41d0-a915-7a501ac6f29a","Type":"ContainerStarted","Data":"45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd"}
Jul 22 15:32:16.264867 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:16.264727    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch" event={"ID":"99ed7f98-651b-41d0-a915-7a501ac6f29a","Type":"ContainerStarted","Data":"5729e0476e8af8dfcd8207c3826a558fdc6698d5651aaf6fd898cf5d80c970b6"}
Jul 22 15:32:16.264867 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:16.264827    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:16.282081 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:16.282036    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch" podStartSLOduration=1.8294206819999999 podStartE2EDuration="2.282022586s" totalImagesPullingTime="452.601904ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:32:16.281111877 +0000 UTC m=+1674.735825494" watchObservedRunningTime="2026-07-22 15:32:16.282022586 +0000 UTC m=+1674.736736200"
Jul 22 15:32:22.276092 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:22.276061    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:39.460645 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:39.460610    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"]
Jul 22 15:32:39.464294 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:39.464272    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"
Jul 22 15:32:39.467676 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:39.467652    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:32:39.473550 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:39.473527    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"]
Jul 22 15:32:39.498785 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:39.498741    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4da378-6a38-4f07-937d-18dc4e24ba6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt\" (UID: \"fa4da378-6a38-4f07-937d-18dc4e24ba6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"
Jul 22 15:32:39.600055 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:39.600012    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4da378-6a38-4f07-937d-18dc4e24ba6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt\" (UID: \"fa4da378-6a38-4f07-937d-18dc4e24ba6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"
Jul 22 15:32:39.600676 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:39.600656    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4da378-6a38-4f07-937d-18dc4e24ba6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt\" (UID: \"fa4da378-6a38-4f07-937d-18dc4e24ba6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"
Jul 22 15:32:39.776587 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:39.776494    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"
Jul 22 15:32:39.913157 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:39.913130    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"]
Jul 22 15:32:39.915396 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:32:39.915368    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa4da378_6a38_4f07_937d_18dc4e24ba6f.slice/crio-27856e39534c1e5d489a7ecaa0ca00b7a7b96cb4d90572a2ada2cdfc5ca6a023 WatchSource:0}: Error finding container 27856e39534c1e5d489a7ecaa0ca00b7a7b96cb4d90572a2ada2cdfc5ca6a023: Status 404 returned error can't find the container with id 27856e39534c1e5d489a7ecaa0ca00b7a7b96cb4d90572a2ada2cdfc5ca6a023
Jul 22 15:32:40.380493 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:40.380431    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt" event={"ID":"fa4da378-6a38-4f07-937d-18dc4e24ba6f","Type":"ContainerStarted","Data":"3a96237a8f2a876c12f817c7579b064d15f391be0b74ed3309104bc77a69c3b5"}
Jul 22 15:32:40.380691 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:40.380512    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt" event={"ID":"fa4da378-6a38-4f07-937d-18dc4e24ba6f","Type":"ContainerStarted","Data":"27856e39534c1e5d489a7ecaa0ca00b7a7b96cb4d90572a2ada2cdfc5ca6a023"}
Jul 22 15:32:40.396282 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:40.396180    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt" podStartSLOduration=1.396159062 podStartE2EDuration="1.396159062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:32:40.394596594 +0000 UTC m=+1698.849310210" watchObservedRunningTime="2026-07-22 15:32:40.396159062 +0000 UTC m=+1698.850872677"
Jul 22 15:32:41.387921 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:41.387887    2560 generic.go:356] "Generic (PLEG): container finished" podID="fa4da378-6a38-4f07-937d-18dc4e24ba6f" containerID="3a96237a8f2a876c12f817c7579b064d15f391be0b74ed3309104bc77a69c3b5" exitCode=0
Jul 22 15:32:41.388319 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:41.387933    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt" event={"ID":"fa4da378-6a38-4f07-937d-18dc4e24ba6f","Type":"ContainerDied","Data":"3a96237a8f2a876c12f817c7579b064d15f391be0b74ed3309104bc77a69c3b5"}
Jul 22 15:32:42.525367 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:42.525343    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"
Jul 22 15:32:42.721919 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:42.721814    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4da378-6a38-4f07-937d-18dc4e24ba6f-maas-api-service-ca\") pod \"fa4da378-6a38-4f07-937d-18dc4e24ba6f\" (UID: \"fa4da378-6a38-4f07-937d-18dc4e24ba6f\") "
Jul 22 15:32:42.722156 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:42.722129    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4da378-6a38-4f07-937d-18dc4e24ba6f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa4da378-6a38-4f07-937d-18dc4e24ba6f" (UID: "fa4da378-6a38-4f07-937d-18dc4e24ba6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:32:42.823228 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:42.823195    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4da378-6a38-4f07-937d-18dc4e24ba6f-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:32:43.399258 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:43.399201    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"
Jul 22 15:32:43.399258 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:43.399221    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt" event={"ID":"fa4da378-6a38-4f07-937d-18dc4e24ba6f","Type":"ContainerDied","Data":"27856e39534c1e5d489a7ecaa0ca00b7a7b96cb4d90572a2ada2cdfc5ca6a023"}
Jul 22 15:32:43.399509 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:43.399281    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27856e39534c1e5d489a7ecaa0ca00b7a7b96cb4d90572a2ada2cdfc5ca6a023"
Jul 22 15:32:50.485963 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.485924    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"]
Jul 22 15:32:50.486497 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.486250    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch" podUID="99ed7f98-651b-41d0-a915-7a501ac6f29a" containerName="maas-api" containerID="cri-o://45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd" gracePeriod=30
Jul 22 15:32:50.740677 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.740605    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:50.798176 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.798141    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99ed7f98-651b-41d0-a915-7a501ac6f29a-maas-api-tls\") pod \"99ed7f98-651b-41d0-a915-7a501ac6f29a\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") "
Jul 22 15:32:50.798373 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.798201    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpbmk\" (UniqueName: \"kubernetes.io/projected/99ed7f98-651b-41d0-a915-7a501ac6f29a-kube-api-access-vpbmk\") pod \"99ed7f98-651b-41d0-a915-7a501ac6f29a\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") "
Jul 22 15:32:50.798373 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.798262    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ed7f98-651b-41d0-a915-7a501ac6f29a-openshift-service-ca\") pod \"99ed7f98-651b-41d0-a915-7a501ac6f29a\" (UID: \"99ed7f98-651b-41d0-a915-7a501ac6f29a\") "
Jul 22 15:32:50.798716 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.798682    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99ed7f98-651b-41d0-a915-7a501ac6f29a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99ed7f98-651b-41d0-a915-7a501ac6f29a" (UID: "99ed7f98-651b-41d0-a915-7a501ac6f29a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:32:50.800324 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.800300    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99ed7f98-651b-41d0-a915-7a501ac6f29a-kube-api-access-vpbmk" (OuterVolumeSpecName: "kube-api-access-vpbmk") pod "99ed7f98-651b-41d0-a915-7a501ac6f29a" (UID: "99ed7f98-651b-41d0-a915-7a501ac6f29a"). InnerVolumeSpecName "kube-api-access-vpbmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:32:50.800429 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.800411    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99ed7f98-651b-41d0-a915-7a501ac6f29a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99ed7f98-651b-41d0-a915-7a501ac6f29a" (UID: "99ed7f98-651b-41d0-a915-7a501ac6f29a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:32:50.899330 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.899293    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99ed7f98-651b-41d0-a915-7a501ac6f29a-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:32:50.899330 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.899324    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpbmk\" (UniqueName: \"kubernetes.io/projected/99ed7f98-651b-41d0-a915-7a501ac6f29a-kube-api-access-vpbmk\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:32:50.899330 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:50.899334    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99ed7f98-651b-41d0-a915-7a501ac6f29a-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:32:51.440191 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:51.440155    2560 generic.go:356] "Generic (PLEG): container finished" podID="99ed7f98-651b-41d0-a915-7a501ac6f29a" containerID="45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd" exitCode=0
Jul 22 15:32:51.440372 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:51.440203    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"
Jul 22 15:32:51.440372 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:51.440233    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch" event={"ID":"99ed7f98-651b-41d0-a915-7a501ac6f29a","Type":"ContainerDied","Data":"45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd"}
Jul 22 15:32:51.440372 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:51.440276    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch" event={"ID":"99ed7f98-651b-41d0-a915-7a501ac6f29a","Type":"ContainerDied","Data":"5729e0476e8af8dfcd8207c3826a558fdc6698d5651aaf6fd898cf5d80c970b6"}
Jul 22 15:32:51.440372 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:51.440297    2560 scope.go:117] "RemoveContainer" containerID="45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd"
Jul 22 15:32:51.450061 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:51.450039    2560 scope.go:117] "RemoveContainer" containerID="45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd"
Jul 22 15:32:51.450333 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:32:51.450311    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd\": container with ID starting with 45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd not found: ID does not exist" containerID="45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd"
Jul 22 15:32:51.450376 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:51.450345    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd"} err="failed to get container status \"45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd\": rpc error: code = NotFound desc = could not find container \"45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd\": container with ID starting with 45da196f63e3cce3f66e5b90c6e6ffab106baa0fb74655185b7ac0eb99de78bd not found: ID does not exist"
Jul 22 15:32:51.461660 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:51.461631    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"]
Jul 22 15:32:51.465602 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:51.465578    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5fe701b2-749546d8b8-cwzch"]
Jul 22 15:32:52.602422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:52.602384    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99ed7f98-651b-41d0-a915-7a501ac6f29a" path="/var/lib/kubelet/pods/99ed7f98-651b-41d0-a915-7a501ac6f29a/volumes"
Jul 22 15:32:54.538146 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:54.538110    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"]
Jul 22 15:32:54.540647 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:54.540621    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5fe701b2-bp2gt"]
Jul 22 15:32:54.602289 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:32:54.602254    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa4da378-6a38-4f07-937d-18dc4e24ba6f" path="/var/lib/kubelet/pods/fa4da378-6a38-4f07-937d-18dc4e24ba6f/volumes"
Jul 22 15:34:22.148552 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:22.148528    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:34:22.155557 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:22.155535    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:34:22.157374 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:22.157355    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:34:22.164259 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:22.164232    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:34:35.669785 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.669747    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"]
Jul 22 15:34:35.670152 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.670121    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa4da378-6a38-4f07-937d-18dc4e24ba6f" containerName="revoke-keys"
Jul 22 15:34:35.670152 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.670133    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa4da378-6a38-4f07-937d-18dc4e24ba6f" containerName="revoke-keys"
Jul 22 15:34:35.670152 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.670145    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99ed7f98-651b-41d0-a915-7a501ac6f29a" containerName="maas-api"
Jul 22 15:34:35.670152 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.670151    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="99ed7f98-651b-41d0-a915-7a501ac6f29a" containerName="maas-api"
Jul 22 15:34:35.670279 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.670231    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa4da378-6a38-4f07-937d-18dc4e24ba6f" containerName="revoke-keys"
Jul 22 15:34:35.670279 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.670240    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="99ed7f98-651b-41d0-a915-7a501ac6f29a" containerName="maas-api"
Jul 22 15:34:35.673426 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.673405    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:35.679888 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.679865    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-41296eff-serving-cert\""
Jul 22 15:34:35.680775 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.680754    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8q48q\""
Jul 22 15:34:35.692050 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.692025    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"]
Jul 22 15:34:35.757549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.757508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-openshift-service-ca\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:35.757720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.757574    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-maas-api-tls\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:35.757720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.757677    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf9xl\" (UniqueName: \"kubernetes.io/projected/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-kube-api-access-vf9xl\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:35.858986 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.858949    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf9xl\" (UniqueName: \"kubernetes.io/projected/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-kube-api-access-vf9xl\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:35.859209 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.859015    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-openshift-service-ca\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:35.859209 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.859057    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-maas-api-tls\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:35.859209 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:34:35.859198    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-serving-cert: secret "maas-api-e2e-mt-41296eff-serving-cert" not found
Jul 22 15:34:35.859646 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:34:35.859272    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-maas-api-tls podName:a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105 nodeName:}" failed. No retries permitted until 2026-07-22 15:34:36.359252099 +0000 UTC m=+1814.813965692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-maas-api-tls") pod "maas-api-e2e-mt-41296eff-855db87cd9-6qkg8" (UID: "a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105") : secret "maas-api-e2e-mt-41296eff-serving-cert" not found
Jul 22 15:34:35.859802 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.859783    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-openshift-service-ca\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:35.870990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:35.870967    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf9xl\" (UniqueName: \"kubernetes.io/projected/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-kube-api-access-vf9xl\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:36.363765 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:36.363724    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-maas-api-tls\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:36.366418 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:36.366382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-maas-api-tls\") pod \"maas-api-e2e-mt-41296eff-855db87cd9-6qkg8\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:36.584381 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:36.584329    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:36.724009 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:36.723962    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"]
Jul 22 15:34:36.726835 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:34:36.726805    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ffe1f7_e7b6_469f_bc7e_0ef7e195d105.slice/crio-e7d91ffbeac91d297a887e167bb02665a7267033292fe9258a52aa1f814a6bde WatchSource:0}: Error finding container e7d91ffbeac91d297a887e167bb02665a7267033292fe9258a52aa1f814a6bde: Status 404 returned error can't find the container with id e7d91ffbeac91d297a887e167bb02665a7267033292fe9258a52aa1f814a6bde
Jul 22 15:34:36.959498 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:36.959458    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8" event={"ID":"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105","Type":"ContainerStarted","Data":"e7d91ffbeac91d297a887e167bb02665a7267033292fe9258a52aa1f814a6bde"}
Jul 22 15:34:37.965812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:37.965765    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8" event={"ID":"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105","Type":"ContainerStarted","Data":"865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24"}
Jul 22 15:34:37.966215 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:37.965874    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:37.984545 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:37.984483    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8" podStartSLOduration=2.48295306 podStartE2EDuration="2.984436216s" totalImagesPullingTime="501.483156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:34:37.98396699 +0000 UTC m=+1816.438680622" watchObservedRunningTime="2026-07-22 15:34:37.984436216 +0000 UTC m=+1816.439149831"
Jul 22 15:34:43.977218 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:43.977186    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:58.117197 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.117153    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"]
Jul 22 15:34:58.117830 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.117422    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8" podUID="a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105" containerName="maas-api" containerID="cri-o://865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24" gracePeriod=30
Jul 22 15:34:58.366670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.366646    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:58.467946 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.467893    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf9xl\" (UniqueName: \"kubernetes.io/projected/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-kube-api-access-vf9xl\") pod \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") "
Jul 22 15:34:58.468142 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.467993    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-maas-api-tls\") pod \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") "
Jul 22 15:34:58.468142 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.468052    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-openshift-service-ca\") pod \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\" (UID: \"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105\") "
Jul 22 15:34:58.468501 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.468431    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105" (UID: "a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:34:58.470117 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.470084    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105" (UID: "a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:34:58.470213 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.470128    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-kube-api-access-vf9xl" (OuterVolumeSpecName: "kube-api-access-vf9xl") pod "a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105" (UID: "a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105"). InnerVolumeSpecName "kube-api-access-vf9xl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:34:58.569318 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.569269    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:34:58.569318 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.569313    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:34:58.569318 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:58.569324    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf9xl\" (UniqueName: \"kubernetes.io/projected/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105-kube-api-access-vf9xl\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:34:59.067080 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:59.067046    2560 generic.go:356] "Generic (PLEG): container finished" podID="a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105" containerID="865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24" exitCode=0
Jul 22 15:34:59.067247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:59.067091    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"
Jul 22 15:34:59.067247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:59.067110    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8" event={"ID":"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105","Type":"ContainerDied","Data":"865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24"}
Jul 22 15:34:59.067247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:59.067152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8" event={"ID":"a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105","Type":"ContainerDied","Data":"e7d91ffbeac91d297a887e167bb02665a7267033292fe9258a52aa1f814a6bde"}
Jul 22 15:34:59.067247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:59.067172    2560 scope.go:117] "RemoveContainer" containerID="865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24"
Jul 22 15:34:59.075633 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:59.075616    2560 scope.go:117] "RemoveContainer" containerID="865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24"
Jul 22 15:34:59.075873 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:34:59.075854    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24\": container with ID starting with 865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24 not found: ID does not exist" containerID="865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24"
Jul 22 15:34:59.075943 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:59.075879    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24"} err="failed to get container status \"865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24\": rpc error: code = NotFound desc = could not find container \"865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24\": container with ID starting with 865d6f4813661c0e055e4b946ba2444453c198beca1d3e0e7c248a4c65779e24 not found: ID does not exist"
Jul 22 15:34:59.087803 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:59.087771    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"]
Jul 22 15:34:59.089478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:34:59.089432    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-41296eff-855db87cd9-6qkg8"]
Jul 22 15:35:00.602052 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:00.602011    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105" path="/var/lib/kubelet/pods/a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105/volumes"
Jul 22 15:35:32.124096 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.124053    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"]
Jul 22 15:35:32.124774 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.124752    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105" containerName="maas-api"
Jul 22 15:35:32.124774 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.124778    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105" containerName="maas-api"
Jul 22 15:35:32.124913 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.124899    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ffe1f7-e7b6-469f-bc7e-0ef7e195d105" containerName="maas-api"
Jul 22 15:35:32.128577 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.128550    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.134938 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.134898    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ca366ee0-serving-cert\""
Jul 22 15:35:32.135123 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.135100    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8q48q\""
Jul 22 15:35:32.144534 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.144501    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"]
Jul 22 15:35:32.287476 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.287410    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fd0dfc-04f8-49e3-97d7-81daa7f37817-openshift-service-ca\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.287683 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.287495    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxqwp\" (UniqueName: \"kubernetes.io/projected/36fd0dfc-04f8-49e3-97d7-81daa7f37817-kube-api-access-lxqwp\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.287683 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.287562    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fd0dfc-04f8-49e3-97d7-81daa7f37817-maas-api-tls\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.389105 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.388998    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fd0dfc-04f8-49e3-97d7-81daa7f37817-maas-api-tls\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.389105 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.389097    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fd0dfc-04f8-49e3-97d7-81daa7f37817-openshift-service-ca\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.389347 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.389141    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxqwp\" (UniqueName: \"kubernetes.io/projected/36fd0dfc-04f8-49e3-97d7-81daa7f37817-kube-api-access-lxqwp\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.389347 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:35:32.389193    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-serving-cert: secret "maas-api-e2e-mt-ca366ee0-serving-cert" not found
Jul 22 15:35:32.389347 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:35:32.389279    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36fd0dfc-04f8-49e3-97d7-81daa7f37817-maas-api-tls podName:36fd0dfc-04f8-49e3-97d7-81daa7f37817 nodeName:}" failed. No retries permitted until 2026-07-22 15:35:32.889257432 +0000 UTC m=+1871.343971029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36fd0dfc-04f8-49e3-97d7-81daa7f37817-maas-api-tls") pod "maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp" (UID: "36fd0dfc-04f8-49e3-97d7-81daa7f37817") : secret "maas-api-e2e-mt-ca366ee0-serving-cert" not found
Jul 22 15:35:32.389823 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.389802    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fd0dfc-04f8-49e3-97d7-81daa7f37817-openshift-service-ca\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.398608 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.398577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxqwp\" (UniqueName: \"kubernetes.io/projected/36fd0dfc-04f8-49e3-97d7-81daa7f37817-kube-api-access-lxqwp\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.894299 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.894264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fd0dfc-04f8-49e3-97d7-81daa7f37817-maas-api-tls\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:32.896707 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:32.896674    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fd0dfc-04f8-49e3-97d7-81daa7f37817-maas-api-tls\") pod \"maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:33.042732 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:33.042685    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:33.196825 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:33.196584    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"]
Jul 22 15:35:33.199831 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:35:33.199801    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36fd0dfc_04f8_49e3_97d7_81daa7f37817.slice/crio-a5a5d87e0644e2fe914de2a036ea8acf7a1dc5c64996a3bc3e66613d8733f13b WatchSource:0}: Error finding container a5a5d87e0644e2fe914de2a036ea8acf7a1dc5c64996a3bc3e66613d8733f13b: Status 404 returned error can't find the container with id a5a5d87e0644e2fe914de2a036ea8acf7a1dc5c64996a3bc3e66613d8733f13b
Jul 22 15:35:33.201163 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:33.201141    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:35:33.239755 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:33.239721    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp" event={"ID":"36fd0dfc-04f8-49e3-97d7-81daa7f37817","Type":"ContainerStarted","Data":"a5a5d87e0644e2fe914de2a036ea8acf7a1dc5c64996a3bc3e66613d8733f13b"}
Jul 22 15:35:34.245850 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:34.245808    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp" event={"ID":"36fd0dfc-04f8-49e3-97d7-81daa7f37817","Type":"ContainerStarted","Data":"6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b"}
Jul 22 15:35:34.246337 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:34.245874    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:34.265467 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:34.265400    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp" podStartSLOduration=1.814636444 podStartE2EDuration="2.265384813s" totalImagesPullingTime="450.748369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:35:34.263511575 +0000 UTC m=+1872.718225191" watchObservedRunningTime="2026-07-22 15:35:34.265384813 +0000 UTC m=+1872.720098428"
Jul 22 15:35:40.256719 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:40.256688    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:35:50.154559 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.154473    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"]
Jul 22 15:35:50.158928 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.158904    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.162774 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.162747    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ca366ee0\"/\"e2e-lifecycle-model-ca366ee0-kserve-self-signed-certs\""
Jul 22 15:35:50.162954 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.162799    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ca366ee0\"/\"openshift-service-ca.crt\""
Jul 22 15:35:50.162954 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.162913    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ca366ee0\"/\"default-dockercfg-bhtx2\""
Jul 22 15:35:50.162954 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.162941    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ca366ee0\"/\"kube-root-ca.crt\""
Jul 22 15:35:50.168875 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.168851    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"]
Jul 22 15:35:50.252281 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.252248    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-home\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.252507 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.252301    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-tmp-dir\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.252507 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.252335    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dba5581b-861c-40e7-bede-50a07a0976eb-tls-certs\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.252507 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.252405    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-dshm\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.252507 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.252442    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg86c\" (UniqueName: \"kubernetes.io/projected/dba5581b-861c-40e7-bede-50a07a0976eb-kube-api-access-wg86c\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.252670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.252539    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-model-cache\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.353168 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.353134    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-model-cache\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.353350 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.353187    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-home\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.353350 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.353243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-tmp-dir\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.353350 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.353281    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dba5581b-861c-40e7-bede-50a07a0976eb-tls-certs\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.353350 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.353327    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-dshm\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.353594 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.353351    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg86c\" (UniqueName: \"kubernetes.io/projected/dba5581b-861c-40e7-bede-50a07a0976eb-kube-api-access-wg86c\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.353656 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.353595    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-home\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.353714 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.353649    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-model-cache\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.353787 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.353770    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-tmp-dir\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.355683 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.355662    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-dshm\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.355873 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.355855    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dba5581b-861c-40e7-bede-50a07a0976eb-tls-certs\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.361950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.361920    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg86c\" (UniqueName: \"kubernetes.io/projected/dba5581b-861c-40e7-bede-50a07a0976eb-kube-api-access-wg86c\") pod \"e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") " pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.471516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.471404    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:35:50.615018 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:50.614982    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"]
Jul 22 15:35:50.618292 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:35:50.618261    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddba5581b_861c_40e7_bede_50a07a0976eb.slice/crio-8a58d997186bb4fa4bc5066e8971b88e9b0f8da72b85b0c2baafc6b8f21d5262 WatchSource:0}: Error finding container 8a58d997186bb4fa4bc5066e8971b88e9b0f8da72b85b0c2baafc6b8f21d5262: Status 404 returned error can't find the container with id 8a58d997186bb4fa4bc5066e8971b88e9b0f8da72b85b0c2baafc6b8f21d5262
Jul 22 15:35:51.338980 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:51.338939    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8" event={"ID":"dba5581b-861c-40e7-bede-50a07a0976eb","Type":"ContainerStarted","Data":"5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4"}
Jul 22 15:35:51.339384 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:51.338993    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8" event={"ID":"dba5581b-861c-40e7-bede-50a07a0976eb","Type":"ContainerStarted","Data":"8a58d997186bb4fa4bc5066e8971b88e9b0f8da72b85b0c2baafc6b8f21d5262"}
Jul 22 15:35:51.359844 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:35:51.359777    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8" podStartSLOduration=1.3597571450000001 podStartE2EDuration="1.359757145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:35:51.359510215 +0000 UTC m=+1889.814223831" watchObservedRunningTime="2026-07-22 15:35:51.359757145 +0000 UTC m=+1889.814470760"
Jul 22 15:36:00.472262 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:00.472222    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:36:00.472262 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:00.472269    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:36:00.485241 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:00.485212    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:36:01.399820 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:01.399791    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:36:35.807710 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:35.807660    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"]
Jul 22 15:36:35.808241 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:35.807897    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp" podUID="36fd0dfc-04f8-49e3-97d7-81daa7f37817" containerName="maas-api" containerID="cri-o://6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b" gracePeriod=30
Jul 22 15:36:36.055221 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.055196    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:36:36.191608 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.191573    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fd0dfc-04f8-49e3-97d7-81daa7f37817-openshift-service-ca\") pod \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") "
Jul 22 15:36:36.191835 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.191714    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fd0dfc-04f8-49e3-97d7-81daa7f37817-maas-api-tls\") pod \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") "
Jul 22 15:36:36.191835 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.191759    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxqwp\" (UniqueName: \"kubernetes.io/projected/36fd0dfc-04f8-49e3-97d7-81daa7f37817-kube-api-access-lxqwp\") pod \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\" (UID: \"36fd0dfc-04f8-49e3-97d7-81daa7f37817\") "
Jul 22 15:36:36.192043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.192010    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36fd0dfc-04f8-49e3-97d7-81daa7f37817-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "36fd0dfc-04f8-49e3-97d7-81daa7f37817" (UID: "36fd0dfc-04f8-49e3-97d7-81daa7f37817"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:36:36.193838 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.193808    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36fd0dfc-04f8-49e3-97d7-81daa7f37817-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "36fd0dfc-04f8-49e3-97d7-81daa7f37817" (UID: "36fd0dfc-04f8-49e3-97d7-81daa7f37817"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:36:36.194050 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.194023    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36fd0dfc-04f8-49e3-97d7-81daa7f37817-kube-api-access-lxqwp" (OuterVolumeSpecName: "kube-api-access-lxqwp") pod "36fd0dfc-04f8-49e3-97d7-81daa7f37817" (UID: "36fd0dfc-04f8-49e3-97d7-81daa7f37817"). InnerVolumeSpecName "kube-api-access-lxqwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:36:36.292875 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.292835    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36fd0dfc-04f8-49e3-97d7-81daa7f37817-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:36:36.292875 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.292868    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxqwp\" (UniqueName: \"kubernetes.io/projected/36fd0dfc-04f8-49e3-97d7-81daa7f37817-kube-api-access-lxqwp\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:36:36.292875 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.292882    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36fd0dfc-04f8-49e3-97d7-81daa7f37817-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:36:36.563813 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.563716    2560 generic.go:356] "Generic (PLEG): container finished" podID="36fd0dfc-04f8-49e3-97d7-81daa7f37817" containerID="6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b" exitCode=0
Jul 22 15:36:36.563813 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.563750    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp" event={"ID":"36fd0dfc-04f8-49e3-97d7-81daa7f37817","Type":"ContainerDied","Data":"6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b"}
Jul 22 15:36:36.563813 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.563808    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp" event={"ID":"36fd0dfc-04f8-49e3-97d7-81daa7f37817","Type":"ContainerDied","Data":"a5a5d87e0644e2fe914de2a036ea8acf7a1dc5c64996a3bc3e66613d8733f13b"}
Jul 22 15:36:36.564138 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.563833    2560 scope.go:117] "RemoveContainer" containerID="6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b"
Jul 22 15:36:36.564138 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.563764    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"
Jul 22 15:36:36.573884 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.573856    2560 scope.go:117] "RemoveContainer" containerID="6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b"
Jul 22 15:36:36.574173 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:36:36.574154    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b\": container with ID starting with 6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b not found: ID does not exist" containerID="6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b"
Jul 22 15:36:36.574244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.574204    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b"} err="failed to get container status \"6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b\": rpc error: code = NotFound desc = could not find container \"6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b\": container with ID starting with 6544398b25320a2dde5fda6f9e4799e705d2b0003a658af29311d8c55e4a283b not found: ID does not exist"
Jul 22 15:36:36.585671 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.585644    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"]
Jul 22 15:36:36.592173 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.592147    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ca366ee0-6d886c76f9-5fmmp"]
Jul 22 15:36:36.601856 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:36.601830    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36fd0dfc-04f8-49e3-97d7-81daa7f37817" path="/var/lib/kubelet/pods/36fd0dfc-04f8-49e3-97d7-81daa7f37817/volumes"
Jul 22 15:36:47.741062 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:36:47.741016    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"]
Jul 22 15:37:02.762188 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:02.762143    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8" podUID="dba5581b-861c-40e7-bede-50a07a0976eb" containerName="main" containerID="cri-o://5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4" gracePeriod=45
Jul 22 15:37:03.023194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.023125    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:37:03.141417 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.141381    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-model-cache\") pod \"dba5581b-861c-40e7-bede-50a07a0976eb\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") "
Jul 22 15:37:03.141633 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.141483    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-home\") pod \"dba5581b-861c-40e7-bede-50a07a0976eb\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") "
Jul 22 15:37:03.141633 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.141552    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-dshm\") pod \"dba5581b-861c-40e7-bede-50a07a0976eb\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") "
Jul 22 15:37:03.141633 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.141589    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-tmp-dir\") pod \"dba5581b-861c-40e7-bede-50a07a0976eb\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") "
Jul 22 15:37:03.141633 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.141611    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dba5581b-861c-40e7-bede-50a07a0976eb-tls-certs\") pod \"dba5581b-861c-40e7-bede-50a07a0976eb\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") "
Jul 22 15:37:03.141846 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.141639    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg86c\" (UniqueName: \"kubernetes.io/projected/dba5581b-861c-40e7-bede-50a07a0976eb-kube-api-access-wg86c\") pod \"dba5581b-861c-40e7-bede-50a07a0976eb\" (UID: \"dba5581b-861c-40e7-bede-50a07a0976eb\") "
Jul 22 15:37:03.141846 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.141674    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-model-cache" (OuterVolumeSpecName: "model-cache") pod "dba5581b-861c-40e7-bede-50a07a0976eb" (UID: "dba5581b-861c-40e7-bede-50a07a0976eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:37:03.141846 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.141783    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-home" (OuterVolumeSpecName: "home") pod "dba5581b-861c-40e7-bede-50a07a0976eb" (UID: "dba5581b-861c-40e7-bede-50a07a0976eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:37:03.141987 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.141872    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dba5581b-861c-40e7-bede-50a07a0976eb" (UID: "dba5581b-861c-40e7-bede-50a07a0976eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:37:03.142087 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.142070    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:37:03.142122 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.142092    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:37:03.142122 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.142105    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:37:03.143858 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.143828    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-dshm" (OuterVolumeSpecName: "dshm") pod "dba5581b-861c-40e7-bede-50a07a0976eb" (UID: "dba5581b-861c-40e7-bede-50a07a0976eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:37:03.143858 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.143844    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba5581b-861c-40e7-bede-50a07a0976eb-kube-api-access-wg86c" (OuterVolumeSpecName: "kube-api-access-wg86c") pod "dba5581b-861c-40e7-bede-50a07a0976eb" (UID: "dba5581b-861c-40e7-bede-50a07a0976eb"). InnerVolumeSpecName "kube-api-access-wg86c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:37:03.144028 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.143868    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba5581b-861c-40e7-bede-50a07a0976eb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dba5581b-861c-40e7-bede-50a07a0976eb" (UID: "dba5581b-861c-40e7-bede-50a07a0976eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:37:03.243505 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.243473    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dba5581b-861c-40e7-bede-50a07a0976eb-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:37:03.243505 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.243500    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dba5581b-861c-40e7-bede-50a07a0976eb-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:37:03.243505 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.243511    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg86c\" (UniqueName: \"kubernetes.io/projected/dba5581b-861c-40e7-bede-50a07a0976eb-kube-api-access-wg86c\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:37:03.704898 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.704863    2560 generic.go:356] "Generic (PLEG): container finished" podID="dba5581b-861c-40e7-bede-50a07a0976eb" containerID="5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4" exitCode=0
Jul 22 15:37:03.704898 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.704901    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"
Jul 22 15:37:03.705114 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.704903    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8" event={"ID":"dba5581b-861c-40e7-bede-50a07a0976eb","Type":"ContainerDied","Data":"5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4"}
Jul 22 15:37:03.705114 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.705024    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8" event={"ID":"dba5581b-861c-40e7-bede-50a07a0976eb","Type":"ContainerDied","Data":"8a58d997186bb4fa4bc5066e8971b88e9b0f8da72b85b0c2baafc6b8f21d5262"}
Jul 22 15:37:03.705114 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.705044    2560 scope.go:117] "RemoveContainer" containerID="5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4"
Jul 22 15:37:03.714471 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.714431    2560 scope.go:117] "RemoveContainer" containerID="5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4"
Jul 22 15:37:03.714741 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:37:03.714720    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4\": container with ID starting with 5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4 not found: ID does not exist" containerID="5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4"
Jul 22 15:37:03.714791 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.714750    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4"} err="failed to get container status \"5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4\": rpc error: code = NotFound desc = could not find container \"5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4\": container with ID starting with 5c5a154ee5cdac4576dbad72ddbf40666edb4b841e2c86487892085e5bab6df4 not found: ID does not exist"
Jul 22 15:37:03.729000 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.728971    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"]
Jul 22 15:37:03.732483 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:03.732276    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ca366ee0/e2e-lifecycle-model-ca366ee0-kserve-65f6fd748c-42nr8"]
Jul 22 15:37:04.602133 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:04.602099    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dba5581b-861c-40e7-bede-50a07a0976eb" path="/var/lib/kubelet/pods/dba5581b-861c-40e7-bede-50a07a0976eb/volumes"
Jul 22 15:37:22.260902 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:22.260824    2560 scope.go:117] "RemoveContainer" containerID="fc6ec3744c3cd332cca86cd66a83ef0b815e78faffb2a7bc1460e7a8893c0d3c"
Jul 22 15:37:22.271305 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:37:22.271281    2560 scope.go:117] "RemoveContainer" containerID="a775024932c77f9f59b34cbe505294cc175065e0c9169a2a92ce1aff7ec3ad5d"
Jul 22 15:38:33.283687 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.283644    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"]
Jul 22 15:38:33.284194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.284012    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba5581b-861c-40e7-bede-50a07a0976eb" containerName="main"
Jul 22 15:38:33.284194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.284022    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba5581b-861c-40e7-bede-50a07a0976eb" containerName="main"
Jul 22 15:38:33.284194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.284043    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36fd0dfc-04f8-49e3-97d7-81daa7f37817" containerName="maas-api"
Jul 22 15:38:33.284194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.284049    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="36fd0dfc-04f8-49e3-97d7-81daa7f37817" containerName="maas-api"
Jul 22 15:38:33.284194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.284111    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba5581b-861c-40e7-bede-50a07a0976eb" containerName="main"
Jul 22 15:38:33.284194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.284121    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="36fd0dfc-04f8-49e3-97d7-81daa7f37817" containerName="maas-api"
Jul 22 15:38:33.287308 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.287288    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:33.292134 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.292111    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-dd89dd-serving-cert\""
Jul 22 15:38:33.292369 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.292351    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8q48q\""
Jul 22 15:38:33.316067 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.316030    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"]
Jul 22 15:38:33.466332 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.466292    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa9b816b-1414-4d4f-bb5a-bb872a89726f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:33.466580 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.466352    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5tx4\" (UniqueName: \"kubernetes.io/projected/fa9b816b-1414-4d4f-bb5a-bb872a89726f-kube-api-access-n5tx4\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:33.466580 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.466497    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa9b816b-1414-4d4f-bb5a-bb872a89726f-maas-api-tls\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:33.567859 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.567769    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa9b816b-1414-4d4f-bb5a-bb872a89726f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:33.567859 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.567837    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5tx4\" (UniqueName: \"kubernetes.io/projected/fa9b816b-1414-4d4f-bb5a-bb872a89726f-kube-api-access-n5tx4\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:33.568087 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.567912    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa9b816b-1414-4d4f-bb5a-bb872a89726f-maas-api-tls\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:33.568087 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:38:33.568034    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-serving-cert: secret "maas-api-e2e-inf-a-dd89dd-serving-cert" not found
Jul 22 15:38:33.568189 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:38:33.568110    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9b816b-1414-4d4f-bb5a-bb872a89726f-maas-api-tls podName:fa9b816b-1414-4d4f-bb5a-bb872a89726f nodeName:}" failed. No retries permitted until 2026-07-22 15:38:34.068086727 +0000 UTC m=+2052.522800321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa9b816b-1414-4d4f-bb5a-bb872a89726f-maas-api-tls") pod "maas-api-e2e-inf-a-dd89dd-757449469c-269qj" (UID: "fa9b816b-1414-4d4f-bb5a-bb872a89726f") : secret "maas-api-e2e-inf-a-dd89dd-serving-cert" not found
Jul 22 15:38:33.568596 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.568577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa9b816b-1414-4d4f-bb5a-bb872a89726f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:33.581272 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:33.581248    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5tx4\" (UniqueName: \"kubernetes.io/projected/fa9b816b-1414-4d4f-bb5a-bb872a89726f-kube-api-access-n5tx4\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:34.072863 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:34.072831    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa9b816b-1414-4d4f-bb5a-bb872a89726f-maas-api-tls\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:34.075443 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:34.075414    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa9b816b-1414-4d4f-bb5a-bb872a89726f-maas-api-tls\") pod \"maas-api-e2e-inf-a-dd89dd-757449469c-269qj\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:34.198649 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:34.198611    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:34.386621 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:34.386547    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"]
Jul 22 15:38:34.397615 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:38:34.397582    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9b816b_1414_4d4f_bb5a_bb872a89726f.slice/crio-fccbff063ee93309dd2452bf9f27a7d35f89fb47aa7617ecf8758dd1a8056679 WatchSource:0}: Error finding container fccbff063ee93309dd2452bf9f27a7d35f89fb47aa7617ecf8758dd1a8056679: Status 404 returned error can't find the container with id fccbff063ee93309dd2452bf9f27a7d35f89fb47aa7617ecf8758dd1a8056679
Jul 22 15:38:35.147491 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:35.147421    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj" event={"ID":"fa9b816b-1414-4d4f-bb5a-bb872a89726f","Type":"ContainerStarted","Data":"d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a"}
Jul 22 15:38:35.147668 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:35.147507    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj" event={"ID":"fa9b816b-1414-4d4f-bb5a-bb872a89726f","Type":"ContainerStarted","Data":"fccbff063ee93309dd2452bf9f27a7d35f89fb47aa7617ecf8758dd1a8056679"}
Jul 22 15:38:35.170774 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:35.170720    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj" podStartSLOduration=1.624578975 podStartE2EDuration="2.170703244s" totalImagesPullingTime="546.124269ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:38:35.168757754 +0000 UTC m=+2053.623471370" watchObservedRunningTime="2026-07-22 15:38:35.170703244 +0000 UTC m=+2053.625416858"
Jul 22 15:38:36.152720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:36.152688    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:41.285343 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.285296    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"]
Jul 22 15:38:41.289661 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.289638    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.292328 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.292306    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-9b66ab-serving-cert\""
Jul 22 15:38:41.298374 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.298350    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"]
Jul 22 15:38:41.329302 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.329266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d871d9c8-14b3-4185-925b-e47915b03fa6-maas-api-tls\") pod \"maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.329302 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.329299    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d871d9c8-14b3-4185-925b-e47915b03fa6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.329555 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.329317    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j6nr\" (UniqueName: \"kubernetes.io/projected/d871d9c8-14b3-4185-925b-e47915b03fa6-kube-api-access-2j6nr\") pod \"maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.429838 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.429803    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d871d9c8-14b3-4185-925b-e47915b03fa6-maas-api-tls\") pod \"maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.429838 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.429844    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d871d9c8-14b3-4185-925b-e47915b03fa6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.430107 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.430026    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j6nr\" (UniqueName: \"kubernetes.io/projected/d871d9c8-14b3-4185-925b-e47915b03fa6-kube-api-access-2j6nr\") pod \"maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.430713 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.430683    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d871d9c8-14b3-4185-925b-e47915b03fa6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.432240 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.432220    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d871d9c8-14b3-4185-925b-e47915b03fa6-maas-api-tls\") pod \"maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.444458 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.444411    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j6nr\" (UniqueName: \"kubernetes.io/projected/d871d9c8-14b3-4185-925b-e47915b03fa6-kube-api-access-2j6nr\") pod \"maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.601291 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.601214    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:41.748956 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:41.748925    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"]
Jul 22 15:38:41.750947 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:38:41.750921    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd871d9c8_14b3_4185_925b_e47915b03fa6.slice/crio-9fad39aa3f91afa749bcdd9d1d926a3f1cf3259ffbd6e05e35796325c2f8082b WatchSource:0}: Error finding container 9fad39aa3f91afa749bcdd9d1d926a3f1cf3259ffbd6e05e35796325c2f8082b: Status 404 returned error can't find the container with id 9fad39aa3f91afa749bcdd9d1d926a3f1cf3259ffbd6e05e35796325c2f8082b
Jul 22 15:38:42.164375 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:42.164342    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:38:42.184955 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:42.184917    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2" event={"ID":"d871d9c8-14b3-4185-925b-e47915b03fa6","Type":"ContainerStarted","Data":"9fad39aa3f91afa749bcdd9d1d926a3f1cf3259ffbd6e05e35796325c2f8082b"}
Jul 22 15:38:43.190970 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:43.190927    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2" event={"ID":"d871d9c8-14b3-4185-925b-e47915b03fa6","Type":"ContainerStarted","Data":"cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b"}
Jul 22 15:38:43.191414 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:43.191095    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:43.218405 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:43.218345    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2" podStartSLOduration=1.603923218 podStartE2EDuration="2.218326853s" totalImagesPullingTime="614.403635ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:38:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:38:43.216426997 +0000 UTC m=+2061.671140628" watchObservedRunningTime="2026-07-22 15:38:43.218326853 +0000 UTC m=+2061.673040468"
Jul 22 15:38:49.202959 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:49.202927    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:38:55.078597 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.078562    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"]
Jul 22 15:38:55.082243 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.082225    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.086751 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.086722    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-dd89dd\"/\"test-model-dd89dd-kserve-self-signed-certs\""
Jul 22 15:38:55.086901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.086722    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-dd89dd\"/\"openshift-service-ca.crt\""
Jul 22 15:38:55.086901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.086763    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-dd89dd\"/\"default-dockercfg-r2sh4\""
Jul 22 15:38:55.086901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.086723    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-dd89dd\"/\"kube-root-ca.crt\""
Jul 22 15:38:55.092576 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.092551    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"]
Jul 22 15:38:55.140002 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.139969    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-model-cache\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.140172 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.140014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tmp-dir\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.140172 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.140034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pddf\" (UniqueName: \"kubernetes.io/projected/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-kube-api-access-4pddf\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.140172 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.140106    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-dshm\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.140385 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.140236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tls-certs\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.140385 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.140292    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-home\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.241292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.241250    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tls-certs\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.241292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.241295    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-home\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.241590 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.241326    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-model-cache\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.241590 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.241349    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tmp-dir\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.241590 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.241367    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pddf\" (UniqueName: \"kubernetes.io/projected/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-kube-api-access-4pddf\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.241590 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.241394    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-dshm\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.241842 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.241819    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-home\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.241995 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.241973    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tmp-dir\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.242063 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.242016    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-model-cache\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.243837 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.243814    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-dshm\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.244070 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.244054    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tls-certs\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.250302 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.250277    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pddf\" (UniqueName: \"kubernetes.io/projected/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-kube-api-access-4pddf\") pod \"test-model-dd89dd-kserve-5d74777967-vjwkh\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") " pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.393121 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.393027    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:38:55.554279 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:55.551909    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"]
Jul 22 15:38:55.557931 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:38:55.557892    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1d9cda8_2e5f_48d4_9c89_65555869cf6a.slice/crio-c29b342284cc7c571c7cd5edbfc0d60e422f08f2d4d6a4a981cd56f94444489a WatchSource:0}: Error finding container c29b342284cc7c571c7cd5edbfc0d60e422f08f2d4d6a4a981cd56f94444489a: Status 404 returned error can't find the container with id c29b342284cc7c571c7cd5edbfc0d60e422f08f2d4d6a4a981cd56f94444489a
Jul 22 15:38:56.262261 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:56.262225    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh" event={"ID":"d1d9cda8-2e5f-48d4-9c89-65555869cf6a","Type":"ContainerStarted","Data":"47675703cdb57964b91d87cc227959d744971213ddf2a2d0ce5789af79e73181"}
Jul 22 15:38:56.262261 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:56.262267    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh" event={"ID":"d1d9cda8-2e5f-48d4-9c89-65555869cf6a","Type":"ContainerStarted","Data":"c29b342284cc7c571c7cd5edbfc0d60e422f08f2d4d6a4a981cd56f94444489a"}
Jul 22 15:38:56.282615 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:38:56.282555    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh" podStartSLOduration=1.282539098 podStartE2EDuration="1.282539098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:38:56.280927163 +0000 UTC m=+2074.735640770" watchObservedRunningTime="2026-07-22 15:38:56.282539098 +0000 UTC m=+2074.737252712"
Jul 22 15:39:05.394161 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:05.394119    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:39:05.394657 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:05.394395    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:39:05.406818 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:05.406793    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:39:06.321595 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:06.321569    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:39:16.380215 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.380180    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"]
Jul 22 15:39:16.384979 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.384956    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.387656 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.387631    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9b66ab\"/\"kube-root-ca.crt\""
Jul 22 15:39:16.388715 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.388688    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9b66ab\"/\"default-dockercfg-v5jl8\""
Jul 22 15:39:16.388715 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.388688    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9b66ab\"/\"openshift-service-ca.crt\""
Jul 22 15:39:16.388933 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.388832    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9b66ab\"/\"test-model-9b66ab-kserve-self-signed-certs\""
Jul 22 15:39:16.397375 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.397347    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"]
Jul 22 15:39:16.537795 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.537762    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-home\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.537795 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.537798    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcf968f8-f9f4-4890-8f36-1026745e753c-tls-certs\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.538069 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.537894    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-dshm\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.538069 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.538003    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-tmp-dir\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.538178 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.538087    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdzhd\" (UniqueName: \"kubernetes.io/projected/dcf968f8-f9f4-4890-8f36-1026745e753c-kube-api-access-wdzhd\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.538178 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.538128    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-model-cache\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.639704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.639596    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-tmp-dir\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.639704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.639670    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdzhd\" (UniqueName: \"kubernetes.io/projected/dcf968f8-f9f4-4890-8f36-1026745e753c-kube-api-access-wdzhd\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.639704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.639706    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-model-cache\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.640004 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.639741    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-home\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.640004 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.639871    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcf968f8-f9f4-4890-8f36-1026745e753c-tls-certs\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.640004 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.639943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-dshm\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.640160 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.640001    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-tmp-dir\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.640160 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.640058    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-model-cache\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.640160 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.640105    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-home\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.642140 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.642119    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-dshm\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.642396 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.642381    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcf968f8-f9f4-4890-8f36-1026745e753c-tls-certs\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.648481 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.648438    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdzhd\" (UniqueName: \"kubernetes.io/projected/dcf968f8-f9f4-4890-8f36-1026745e753c-kube-api-access-wdzhd\") pod \"test-model-9b66ab-kserve-7746cc8cb9-9mzg4\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") " pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.696284 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.696229    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:16.842323 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:16.842297    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"]
Jul 22 15:39:16.844685 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:39:16.844656    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcf968f8_f9f4_4890_8f36_1026745e753c.slice/crio-02e469adb3778159154bb178c484ec5dfc258f23cccfdf47b7ad533945a5a452 WatchSource:0}: Error finding container 02e469adb3778159154bb178c484ec5dfc258f23cccfdf47b7ad533945a5a452: Status 404 returned error can't find the container with id 02e469adb3778159154bb178c484ec5dfc258f23cccfdf47b7ad533945a5a452
Jul 22 15:39:17.367163 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:17.367060    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4" event={"ID":"dcf968f8-f9f4-4890-8f36-1026745e753c","Type":"ContainerStarted","Data":"003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891"}
Jul 22 15:39:17.367163 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:17.367103    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4" event={"ID":"dcf968f8-f9f4-4890-8f36-1026745e753c","Type":"ContainerStarted","Data":"02e469adb3778159154bb178c484ec5dfc258f23cccfdf47b7ad533945a5a452"}
Jul 22 15:39:17.387698 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:17.387641    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4" podStartSLOduration=1.387625156 podStartE2EDuration="1.387625156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:39:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:39:17.386903074 +0000 UTC m=+2095.841616689" watchObservedRunningTime="2026-07-22 15:39:17.387625156 +0000 UTC m=+2095.842338770"
Jul 22 15:39:22.179733 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:22.179626    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:39:22.190188 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:22.187104    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:39:22.196045 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:22.196022    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:39:22.202785 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:22.202765    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:39:22.308969 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:22.308939    2560 scope.go:117] "RemoveContainer" containerID="3a96237a8f2a876c12f817c7579b064d15f391be0b74ed3309104bc77a69c3b5"
Jul 22 15:39:26.696491 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:26.696429    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:26.696901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:26.696507    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:26.708999 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:26.708967    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:39:27.431658 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:39:27.431627    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:40:10.453999 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:10.453947    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"]
Jul 22 15:40:11.380242 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:11.380207    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"]
Jul 22 15:40:21.942675 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:21.942591    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"]
Jul 22 15:40:21.943245 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:21.942851    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj" podUID="fa9b816b-1414-4d4f-bb5a-bb872a89726f" containerName="maas-api" containerID="cri-o://d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a" gracePeriod=30
Jul 22 15:40:22.195119 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.195058    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:40:22.266215 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.266184    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5tx4\" (UniqueName: \"kubernetes.io/projected/fa9b816b-1414-4d4f-bb5a-bb872a89726f-kube-api-access-n5tx4\") pod \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") "
Jul 22 15:40:22.266215 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.266216    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa9b816b-1414-4d4f-bb5a-bb872a89726f-maas-api-tls\") pod \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") "
Jul 22 15:40:22.266407 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.266264    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa9b816b-1414-4d4f-bb5a-bb872a89726f-openshift-service-ca\") pod \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\" (UID: \"fa9b816b-1414-4d4f-bb5a-bb872a89726f\") "
Jul 22 15:40:22.266728 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.266702    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa9b816b-1414-4d4f-bb5a-bb872a89726f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa9b816b-1414-4d4f-bb5a-bb872a89726f" (UID: "fa9b816b-1414-4d4f-bb5a-bb872a89726f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:40:22.268384 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.268360    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa9b816b-1414-4d4f-bb5a-bb872a89726f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa9b816b-1414-4d4f-bb5a-bb872a89726f" (UID: "fa9b816b-1414-4d4f-bb5a-bb872a89726f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:40:22.268492 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.268474    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa9b816b-1414-4d4f-bb5a-bb872a89726f-kube-api-access-n5tx4" (OuterVolumeSpecName: "kube-api-access-n5tx4") pod "fa9b816b-1414-4d4f-bb5a-bb872a89726f" (UID: "fa9b816b-1414-4d4f-bb5a-bb872a89726f"). InnerVolumeSpecName "kube-api-access-n5tx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:40:22.367354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.367298    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5tx4\" (UniqueName: \"kubernetes.io/projected/fa9b816b-1414-4d4f-bb5a-bb872a89726f-kube-api-access-n5tx4\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:22.367354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.367347    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa9b816b-1414-4d4f-bb5a-bb872a89726f-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:22.367354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.367360    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa9b816b-1414-4d4f-bb5a-bb872a89726f-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:22.684513 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.684478    2560 generic.go:356] "Generic (PLEG): container finished" podID="fa9b816b-1414-4d4f-bb5a-bb872a89726f" containerID="d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a" exitCode=0
Jul 22 15:40:22.684678 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.684537    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"
Jul 22 15:40:22.684678 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.684568    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj" event={"ID":"fa9b816b-1414-4d4f-bb5a-bb872a89726f","Type":"ContainerDied","Data":"d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a"}
Jul 22 15:40:22.684678 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.684621    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj" event={"ID":"fa9b816b-1414-4d4f-bb5a-bb872a89726f","Type":"ContainerDied","Data":"fccbff063ee93309dd2452bf9f27a7d35f89fb47aa7617ecf8758dd1a8056679"}
Jul 22 15:40:22.684678 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.684650    2560 scope.go:117] "RemoveContainer" containerID="d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a"
Jul 22 15:40:22.693660 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.693643    2560 scope.go:117] "RemoveContainer" containerID="d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a"
Jul 22 15:40:22.693925 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:40:22.693905    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a\": container with ID starting with d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a not found: ID does not exist" containerID="d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a"
Jul 22 15:40:22.693989 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.693938    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a"} err="failed to get container status \"d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a\": rpc error: code = NotFound desc = could not find container \"d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a\": container with ID starting with d8d15dee7b98958c075816befeffef3464c1565fc0bdf22ba86c8a41da9b856a not found: ID does not exist"
Jul 22 15:40:22.702687 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.702665    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"]
Jul 22 15:40:22.706571 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:22.706550    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj"]
Jul 22 15:40:24.602580 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:24.602544    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa9b816b-1414-4d4f-bb5a-bb872a89726f" path="/var/lib/kubelet/pods/fa9b816b-1414-4d4f-bb5a-bb872a89726f/volumes"
Jul 22 15:40:25.159170 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.159131    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd89dd-757449469c-269qj" podUID="fa9b816b-1414-4d4f-bb5a-bb872a89726f" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.47:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 22 15:40:25.474757 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.474661    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh" podUID="d1d9cda8-2e5f-48d4-9c89-65555869cf6a" containerName="main" containerID="cri-o://47675703cdb57964b91d87cc227959d744971213ddf2a2d0ce5789af79e73181" gracePeriod=45
Jul 22 15:40:25.701761 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.701730    2560 generic.go:356] "Generic (PLEG): container finished" podID="d1d9cda8-2e5f-48d4-9c89-65555869cf6a" containerID="47675703cdb57964b91d87cc227959d744971213ddf2a2d0ce5789af79e73181" exitCode=0
Jul 22 15:40:25.702223 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.701799    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh" event={"ID":"d1d9cda8-2e5f-48d4-9c89-65555869cf6a","Type":"ContainerDied","Data":"47675703cdb57964b91d87cc227959d744971213ddf2a2d0ce5789af79e73181"}
Jul 22 15:40:25.736430 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.736374    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:40:25.797016 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.796978    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pddf\" (UniqueName: \"kubernetes.io/projected/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-kube-api-access-4pddf\") pod \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") "
Jul 22 15:40:25.797199 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797038    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tmp-dir\") pod \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") "
Jul 22 15:40:25.797199 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797184    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-model-cache\") pod \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") "
Jul 22 15:40:25.797292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797228    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tls-certs\") pod \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") "
Jul 22 15:40:25.797292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797259    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-home\") pod \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") "
Jul 22 15:40:25.797370 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797288    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1d9cda8-2e5f-48d4-9c89-65555869cf6a" (UID: "d1d9cda8-2e5f-48d4-9c89-65555869cf6a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:40:25.797370 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797349    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-dshm\") pod \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\" (UID: \"d1d9cda8-2e5f-48d4-9c89-65555869cf6a\") "
Jul 22 15:40:25.797480 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797393    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1d9cda8-2e5f-48d4-9c89-65555869cf6a" (UID: "d1d9cda8-2e5f-48d4-9c89-65555869cf6a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:40:25.797621 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797584    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-home" (OuterVolumeSpecName: "home") pod "d1d9cda8-2e5f-48d4-9c89-65555869cf6a" (UID: "d1d9cda8-2e5f-48d4-9c89-65555869cf6a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:40:25.797721 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797702    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:25.797778 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797722    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:25.797778 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.797737    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:25.799530 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.799503    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-kube-api-access-4pddf" (OuterVolumeSpecName: "kube-api-access-4pddf") pod "d1d9cda8-2e5f-48d4-9c89-65555869cf6a" (UID: "d1d9cda8-2e5f-48d4-9c89-65555869cf6a"). InnerVolumeSpecName "kube-api-access-4pddf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:40:25.799530 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.799505    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1d9cda8-2e5f-48d4-9c89-65555869cf6a" (UID: "d1d9cda8-2e5f-48d4-9c89-65555869cf6a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:40:25.799708 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.799609    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-dshm" (OuterVolumeSpecName: "dshm") pod "d1d9cda8-2e5f-48d4-9c89-65555869cf6a" (UID: "d1d9cda8-2e5f-48d4-9c89-65555869cf6a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:40:25.899030 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.898996    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:25.899030 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.899021    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:25.899030 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:25.899030    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pddf\" (UniqueName: \"kubernetes.io/projected/d1d9cda8-2e5f-48d4-9c89-65555869cf6a-kube-api-access-4pddf\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:26.400719 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.400671    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4" podUID="dcf968f8-f9f4-4890-8f36-1026745e753c" containerName="main" containerID="cri-o://003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891" gracePeriod=45
Jul 22 15:40:26.661761 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.661692    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:40:26.704465 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.704425    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-tmp-dir\") pod \"dcf968f8-f9f4-4890-8f36-1026745e753c\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") "
Jul 22 15:40:26.704918 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.704509    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-home\") pod \"dcf968f8-f9f4-4890-8f36-1026745e753c\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") "
Jul 22 15:40:26.704918 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.704548    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-dshm\") pod \"dcf968f8-f9f4-4890-8f36-1026745e753c\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") "
Jul 22 15:40:26.704918 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.704576    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdzhd\" (UniqueName: \"kubernetes.io/projected/dcf968f8-f9f4-4890-8f36-1026745e753c-kube-api-access-wdzhd\") pod \"dcf968f8-f9f4-4890-8f36-1026745e753c\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") "
Jul 22 15:40:26.704918 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.704665    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcf968f8-f9f4-4890-8f36-1026745e753c-tls-certs\") pod \"dcf968f8-f9f4-4890-8f36-1026745e753c\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") "
Jul 22 15:40:26.704918 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.704711    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dcf968f8-f9f4-4890-8f36-1026745e753c" (UID: "dcf968f8-f9f4-4890-8f36-1026745e753c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:40:26.704918 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.704731    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-model-cache\") pod \"dcf968f8-f9f4-4890-8f36-1026745e753c\" (UID: \"dcf968f8-f9f4-4890-8f36-1026745e753c\") "
Jul 22 15:40:26.705256 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.704953    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:26.705256 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.705148    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-model-cache" (OuterVolumeSpecName: "model-cache") pod "dcf968f8-f9f4-4890-8f36-1026745e753c" (UID: "dcf968f8-f9f4-4890-8f36-1026745e753c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:40:26.705421 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.705380    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-home" (OuterVolumeSpecName: "home") pod "dcf968f8-f9f4-4890-8f36-1026745e753c" (UID: "dcf968f8-f9f4-4890-8f36-1026745e753c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:40:26.707392 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.707353    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcf968f8-f9f4-4890-8f36-1026745e753c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dcf968f8-f9f4-4890-8f36-1026745e753c" (UID: "dcf968f8-f9f4-4890-8f36-1026745e753c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:40:26.707951 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.707922    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-dshm" (OuterVolumeSpecName: "dshm") pod "dcf968f8-f9f4-4890-8f36-1026745e753c" (UID: "dcf968f8-f9f4-4890-8f36-1026745e753c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:40:26.708053 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.708031    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcf968f8-f9f4-4890-8f36-1026745e753c-kube-api-access-wdzhd" (OuterVolumeSpecName: "kube-api-access-wdzhd") pod "dcf968f8-f9f4-4890-8f36-1026745e753c" (UID: "dcf968f8-f9f4-4890-8f36-1026745e753c"). InnerVolumeSpecName "kube-api-access-wdzhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:40:26.711295 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.711267    2560 generic.go:356] "Generic (PLEG): container finished" podID="dcf968f8-f9f4-4890-8f36-1026745e753c" containerID="003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891" exitCode=0
Jul 22 15:40:26.711502 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.711483    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4" event={"ID":"dcf968f8-f9f4-4890-8f36-1026745e753c","Type":"ContainerDied","Data":"003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891"}
Jul 22 15:40:26.711575 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.711525    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4" event={"ID":"dcf968f8-f9f4-4890-8f36-1026745e753c","Type":"ContainerDied","Data":"02e469adb3778159154bb178c484ec5dfc258f23cccfdf47b7ad533945a5a452"}
Jul 22 15:40:26.711575 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.711548    2560 scope.go:117] "RemoveContainer" containerID="003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891"
Jul 22 15:40:26.711672 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.711636    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"
Jul 22 15:40:26.714023 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.713998    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh" event={"ID":"d1d9cda8-2e5f-48d4-9c89-65555869cf6a","Type":"ContainerDied","Data":"c29b342284cc7c571c7cd5edbfc0d60e422f08f2d4d6a4a981cd56f94444489a"}
Jul 22 15:40:26.714133 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.714111    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"
Jul 22 15:40:26.732099 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.732075    2560 scope.go:117] "RemoveContainer" containerID="003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891"
Jul 22 15:40:26.732507 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:40:26.732476    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891\": container with ID starting with 003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891 not found: ID does not exist" containerID="003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891"
Jul 22 15:40:26.732585 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.732520    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891"} err="failed to get container status \"003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891\": rpc error: code = NotFound desc = could not find container \"003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891\": container with ID starting with 003130fb5b989b2a880699c82f9e6a7741a1f98fc335de62ee6c4c2f30c41891 not found: ID does not exist"
Jul 22 15:40:26.732585 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.732540    2560 scope.go:117] "RemoveContainer" containerID="47675703cdb57964b91d87cc227959d744971213ddf2a2d0ce5789af79e73181"
Jul 22 15:40:26.744312 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.744286    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"]
Jul 22 15:40:26.749210 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.749187    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-dd89dd/test-model-dd89dd-kserve-5d74777967-vjwkh"]
Jul 22 15:40:26.761618 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.761594    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"]
Jul 22 15:40:26.768381 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.768356    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9b66ab/test-model-9b66ab-kserve-7746cc8cb9-9mzg4"]
Jul 22 15:40:26.806094 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.806055    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:26.806094 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.806080    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:26.806094 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.806091    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdzhd\" (UniqueName: \"kubernetes.io/projected/dcf968f8-f9f4-4890-8f36-1026745e753c-kube-api-access-wdzhd\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:26.806094 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.806101    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcf968f8-f9f4-4890-8f36-1026745e753c-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:26.806348 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:26.806113    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcf968f8-f9f4-4890-8f36-1026745e753c-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:28.602337 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:28.602302    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d9cda8-2e5f-48d4-9c89-65555869cf6a" path="/var/lib/kubelet/pods/d1d9cda8-2e5f-48d4-9c89-65555869cf6a/volumes"
Jul 22 15:40:28.602817 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:28.602736    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcf968f8-f9f4-4890-8f36-1026745e753c" path="/var/lib/kubelet/pods/dcf968f8-f9f4-4890-8f36-1026745e753c/volumes"
Jul 22 15:40:33.564913 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.564874    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"]
Jul 22 15:40:33.565290 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.565268    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcf968f8-f9f4-4890-8f36-1026745e753c" containerName="main"
Jul 22 15:40:33.565290 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.565279    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcf968f8-f9f4-4890-8f36-1026745e753c" containerName="main"
Jul 22 15:40:33.565363 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.565292    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa9b816b-1414-4d4f-bb5a-bb872a89726f" containerName="maas-api"
Jul 22 15:40:33.565363 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.565298    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa9b816b-1414-4d4f-bb5a-bb872a89726f" containerName="maas-api"
Jul 22 15:40:33.565363 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.565313    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d9cda8-2e5f-48d4-9c89-65555869cf6a" containerName="main"
Jul 22 15:40:33.565363 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.565318    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d9cda8-2e5f-48d4-9c89-65555869cf6a" containerName="main"
Jul 22 15:40:33.565505 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.565376    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa9b816b-1414-4d4f-bb5a-bb872a89726f" containerName="maas-api"
Jul 22 15:40:33.565505 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.565385    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcf968f8-f9f4-4890-8f36-1026745e753c" containerName="main"
Jul 22 15:40:33.565505 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.565394    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d9cda8-2e5f-48d4-9c89-65555869cf6a" containerName="main"
Jul 22 15:40:33.570135 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.570116    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"
Jul 22 15:40:33.573028 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.572999    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:40:33.575431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.575404    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"]
Jul 22 15:40:33.664490 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.664437    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96856c1a-11f3-493a-8b13-1addd069b72e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p\" (UID: \"96856c1a-11f3-493a-8b13-1addd069b72e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"
Jul 22 15:40:33.765838 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.765801    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96856c1a-11f3-493a-8b13-1addd069b72e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p\" (UID: \"96856c1a-11f3-493a-8b13-1addd069b72e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"
Jul 22 15:40:33.766440 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.766417    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96856c1a-11f3-493a-8b13-1addd069b72e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p\" (UID: \"96856c1a-11f3-493a-8b13-1addd069b72e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"
Jul 22 15:40:33.883708 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:33.883617    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"
Jul 22 15:40:34.020510 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:34.020480    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"]
Jul 22 15:40:34.023319 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:40:34.023281    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96856c1a_11f3_493a_8b13_1addd069b72e.slice/crio-4aede457c860edc1a71bdb44b2d85382d75dfcef4ca3aaa466ddf99d34afe36e WatchSource:0}: Error finding container 4aede457c860edc1a71bdb44b2d85382d75dfcef4ca3aaa466ddf99d34afe36e: Status 404 returned error can't find the container with id 4aede457c860edc1a71bdb44b2d85382d75dfcef4ca3aaa466ddf99d34afe36e
Jul 22 15:40:34.753649 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:34.753606    2560 generic.go:356] "Generic (PLEG): container finished" podID="96856c1a-11f3-493a-8b13-1addd069b72e" containerID="cd6318359b5dcba17a1a084cbfa32c82783ff56799c6e063a4317edeb83bcac8" exitCode=0
Jul 22 15:40:34.754040 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:34.753653    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p" event={"ID":"96856c1a-11f3-493a-8b13-1addd069b72e","Type":"ContainerDied","Data":"cd6318359b5dcba17a1a084cbfa32c82783ff56799c6e063a4317edeb83bcac8"}
Jul 22 15:40:34.754040 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:34.753721    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p" event={"ID":"96856c1a-11f3-493a-8b13-1addd069b72e","Type":"ContainerStarted","Data":"4aede457c860edc1a71bdb44b2d85382d75dfcef4ca3aaa466ddf99d34afe36e"}
Jul 22 15:40:35.900100 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:35.900068    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"
Jul 22 15:40:35.987945 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:35.987907    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96856c1a-11f3-493a-8b13-1addd069b72e-maas-api-service-ca\") pod \"96856c1a-11f3-493a-8b13-1addd069b72e\" (UID: \"96856c1a-11f3-493a-8b13-1addd069b72e\") "
Jul 22 15:40:35.988265 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:35.988241    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96856c1a-11f3-493a-8b13-1addd069b72e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96856c1a-11f3-493a-8b13-1addd069b72e" (UID: "96856c1a-11f3-493a-8b13-1addd069b72e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:40:36.088704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:36.088611    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96856c1a-11f3-493a-8b13-1addd069b72e-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:36.764769 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:36.764740    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"
Jul 22 15:40:36.764938 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:36.764767    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p" event={"ID":"96856c1a-11f3-493a-8b13-1addd069b72e","Type":"ContainerDied","Data":"4aede457c860edc1a71bdb44b2d85382d75dfcef4ca3aaa466ddf99d34afe36e"}
Jul 22 15:40:36.764938 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:36.764813    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4aede457c860edc1a71bdb44b2d85382d75dfcef4ca3aaa466ddf99d34afe36e"
Jul 22 15:40:44.120651 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.120619    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"]
Jul 22 15:40:44.121107 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.120816    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2" podUID="d871d9c8-14b3-4185-925b-e47915b03fa6" containerName="maas-api" containerID="cri-o://cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b" gracePeriod=30
Jul 22 15:40:44.197101 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.197065    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2" podUID="d871d9c8-14b3-4185-925b-e47915b03fa6" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.48:8443/health\": dial tcp 10.132.0.48:8443: connect: connection refused"
Jul 22 15:40:44.370028 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.370000    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:40:44.466272 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.466234    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j6nr\" (UniqueName: \"kubernetes.io/projected/d871d9c8-14b3-4185-925b-e47915b03fa6-kube-api-access-2j6nr\") pod \"d871d9c8-14b3-4185-925b-e47915b03fa6\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") "
Jul 22 15:40:44.466483 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.466305    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d871d9c8-14b3-4185-925b-e47915b03fa6-maas-api-tls\") pod \"d871d9c8-14b3-4185-925b-e47915b03fa6\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") "
Jul 22 15:40:44.466483 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.466348    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d871d9c8-14b3-4185-925b-e47915b03fa6-openshift-service-ca\") pod \"d871d9c8-14b3-4185-925b-e47915b03fa6\" (UID: \"d871d9c8-14b3-4185-925b-e47915b03fa6\") "
Jul 22 15:40:44.466788 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.466747    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d871d9c8-14b3-4185-925b-e47915b03fa6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d871d9c8-14b3-4185-925b-e47915b03fa6" (UID: "d871d9c8-14b3-4185-925b-e47915b03fa6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:40:44.468518 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.468486    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d871d9c8-14b3-4185-925b-e47915b03fa6-kube-api-access-2j6nr" (OuterVolumeSpecName: "kube-api-access-2j6nr") pod "d871d9c8-14b3-4185-925b-e47915b03fa6" (UID: "d871d9c8-14b3-4185-925b-e47915b03fa6"). InnerVolumeSpecName "kube-api-access-2j6nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:40:44.468639 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.468514    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d871d9c8-14b3-4185-925b-e47915b03fa6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d871d9c8-14b3-4185-925b-e47915b03fa6" (UID: "d871d9c8-14b3-4185-925b-e47915b03fa6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:40:44.568033 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.567993    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j6nr\" (UniqueName: \"kubernetes.io/projected/d871d9c8-14b3-4185-925b-e47915b03fa6-kube-api-access-2j6nr\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:44.568033 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.568023    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d871d9c8-14b3-4185-925b-e47915b03fa6-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:44.568033 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.568032    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d871d9c8-14b3-4185-925b-e47915b03fa6-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:40:44.805132 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.805041    2560 generic.go:356] "Generic (PLEG): container finished" podID="d871d9c8-14b3-4185-925b-e47915b03fa6" containerID="cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b" exitCode=0
Jul 22 15:40:44.805132 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.805086    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"
Jul 22 15:40:44.805314 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.805128    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2" event={"ID":"d871d9c8-14b3-4185-925b-e47915b03fa6","Type":"ContainerDied","Data":"cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b"}
Jul 22 15:40:44.805314 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.805182    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2" event={"ID":"d871d9c8-14b3-4185-925b-e47915b03fa6","Type":"ContainerDied","Data":"9fad39aa3f91afa749bcdd9d1d926a3f1cf3259ffbd6e05e35796325c2f8082b"}
Jul 22 15:40:44.805314 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.805211    2560 scope.go:117] "RemoveContainer" containerID="cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b"
Jul 22 15:40:44.813940 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.813899    2560 scope.go:117] "RemoveContainer" containerID="cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b"
Jul 22 15:40:44.814211 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:40:44.814187    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b\": container with ID starting with cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b not found: ID does not exist" containerID="cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b"
Jul 22 15:40:44.814284 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.814222    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b"} err="failed to get container status \"cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b\": rpc error: code = NotFound desc = could not find container \"cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b\": container with ID starting with cfbabe5f5d0e4e8bdd56c860d99a6a1ddfdafce38bcddd5a0b57b6b9e858a31b not found: ID does not exist"
Jul 22 15:40:44.823670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.823649    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"]
Jul 22 15:40:44.827229 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:44.827208    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9b66ab-78f8449d4f-xv5d2"]
Jul 22 15:40:46.609056 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:46.609015    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d871d9c8-14b3-4185-925b-e47915b03fa6" path="/var/lib/kubelet/pods/d871d9c8-14b3-4185-925b-e47915b03fa6/volumes"
Jul 22 15:40:48.646007 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:48.645974    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"]
Jul 22 15:40:48.648412 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:48.648389    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b66ab-fkp9p"]
Jul 22 15:40:50.605208 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:50.605176    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96856c1a-11f3-493a-8b13-1addd069b72e" path="/var/lib/kubelet/pods/96856c1a-11f3-493a-8b13-1addd069b72e/volumes"
Jul 22 15:40:59.751029 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.750993    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"]
Jul 22 15:40:59.751440 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.751373    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d871d9c8-14b3-4185-925b-e47915b03fa6" containerName="maas-api"
Jul 22 15:40:59.751440 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.751383    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d871d9c8-14b3-4185-925b-e47915b03fa6" containerName="maas-api"
Jul 22 15:40:59.751440 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.751400    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96856c1a-11f3-493a-8b13-1addd069b72e" containerName="revoke-keys"
Jul 22 15:40:59.751440 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.751405    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="96856c1a-11f3-493a-8b13-1addd069b72e" containerName="revoke-keys"
Jul 22 15:40:59.751635 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.751494    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="96856c1a-11f3-493a-8b13-1addd069b72e" containerName="revoke-keys"
Jul 22 15:40:59.751635 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.751508    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d871d9c8-14b3-4185-925b-e47915b03fa6" containerName="maas-api"
Jul 22 15:40:59.754655 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.754638    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:40:59.757607 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.757582    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8q48q\""
Jul 22 15:40:59.757739 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.757619    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-81c749-serving-cert\""
Jul 22 15:40:59.768525 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.768499    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"]
Jul 22 15:40:59.908720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.908681    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa6c338-3504-488d-99df-66addde0b82e-openshift-service-ca\") pod \"maas-api-e2e-api-a-81c749-69b96dc9c-z88zg\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:40:59.908897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.908740    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4jlr\" (UniqueName: \"kubernetes.io/projected/4aa6c338-3504-488d-99df-66addde0b82e-kube-api-access-x4jlr\") pod \"maas-api-e2e-api-a-81c749-69b96dc9c-z88zg\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:40:59.908897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:40:59.908795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa6c338-3504-488d-99df-66addde0b82e-maas-api-tls\") pod \"maas-api-e2e-api-a-81c749-69b96dc9c-z88zg\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:00.010479 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.010378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa6c338-3504-488d-99df-66addde0b82e-openshift-service-ca\") pod \"maas-api-e2e-api-a-81c749-69b96dc9c-z88zg\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:00.010479 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.010434    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4jlr\" (UniqueName: \"kubernetes.io/projected/4aa6c338-3504-488d-99df-66addde0b82e-kube-api-access-x4jlr\") pod \"maas-api-e2e-api-a-81c749-69b96dc9c-z88zg\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:00.010684 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.010493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa6c338-3504-488d-99df-66addde0b82e-maas-api-tls\") pod \"maas-api-e2e-api-a-81c749-69b96dc9c-z88zg\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:00.011074 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.011054    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa6c338-3504-488d-99df-66addde0b82e-openshift-service-ca\") pod \"maas-api-e2e-api-a-81c749-69b96dc9c-z88zg\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:00.013063 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.013038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa6c338-3504-488d-99df-66addde0b82e-maas-api-tls\") pod \"maas-api-e2e-api-a-81c749-69b96dc9c-z88zg\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:00.023945 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.023912    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4jlr\" (UniqueName: \"kubernetes.io/projected/4aa6c338-3504-488d-99df-66addde0b82e-kube-api-access-x4jlr\") pod \"maas-api-e2e-api-a-81c749-69b96dc9c-z88zg\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:00.065109 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.065078    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:00.205734 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.205706    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"]
Jul 22 15:41:00.207585 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:41:00.207558    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa6c338_3504_488d_99df_66addde0b82e.slice/crio-682beb1883834b3af4a47925c503c6086e174096d6da4ae5916402858f395be3 WatchSource:0}: Error finding container 682beb1883834b3af4a47925c503c6086e174096d6da4ae5916402858f395be3: Status 404 returned error can't find the container with id 682beb1883834b3af4a47925c503c6086e174096d6da4ae5916402858f395be3
Jul 22 15:41:00.209416 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.209392    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:41:00.891111 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.891005    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg" event={"ID":"4aa6c338-3504-488d-99df-66addde0b82e","Type":"ContainerStarted","Data":"9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574"}
Jul 22 15:41:00.891111 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.891086    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg" event={"ID":"4aa6c338-3504-488d-99df-66addde0b82e","Type":"ContainerStarted","Data":"682beb1883834b3af4a47925c503c6086e174096d6da4ae5916402858f395be3"}
Jul 22 15:41:00.891639 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.891123    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:00.911543 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:00.911491    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg" podStartSLOduration=1.486400455 podStartE2EDuration="1.911477006s" totalImagesPullingTime="425.076551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:41:00.909539863 +0000 UTC m=+2199.364253480" watchObservedRunningTime="2026-07-22 15:41:00.911477006 +0000 UTC m=+2199.366190620"
Jul 22 15:41:06.900624 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:06.900590    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:07.867903 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.867864    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"]
Jul 22 15:41:07.871652 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.871634    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:07.880468 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.880426    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b47c9e-serving-cert\""
Jul 22 15:41:07.882438 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.882418    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfnth\" (UniqueName: \"kubernetes.io/projected/b00f8ffd-908f-4b0d-b739-0605cbf935e9-kube-api-access-nfnth\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:07.882523 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.882511    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00f8ffd-908f-4b0d-b739-0605cbf935e9-maas-api-tls\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:07.882576 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.882564    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00f8ffd-908f-4b0d-b739-0605cbf935e9-openshift-service-ca\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:07.900268 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.900234    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"]
Jul 22 15:41:07.984015 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.983966    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00f8ffd-908f-4b0d-b739-0605cbf935e9-maas-api-tls\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:07.984435 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.984047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00f8ffd-908f-4b0d-b739-0605cbf935e9-openshift-service-ca\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:07.984435 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.984114    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfnth\" (UniqueName: \"kubernetes.io/projected/b00f8ffd-908f-4b0d-b739-0605cbf935e9-kube-api-access-nfnth\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:07.984435 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:41:07.984133    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-serving-cert: secret "maas-api-e2e-api-b-b47c9e-serving-cert" not found
Jul 22 15:41:07.984435 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:41:07.984211    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00f8ffd-908f-4b0d-b739-0605cbf935e9-maas-api-tls podName:b00f8ffd-908f-4b0d-b739-0605cbf935e9 nodeName:}" failed. No retries permitted until 2026-07-22 15:41:08.484188711 +0000 UTC m=+2206.938902317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b00f8ffd-908f-4b0d-b739-0605cbf935e9-maas-api-tls") pod "maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7" (UID: "b00f8ffd-908f-4b0d-b739-0605cbf935e9") : secret "maas-api-e2e-api-b-b47c9e-serving-cert" not found
Jul 22 15:41:07.984752 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.984730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00f8ffd-908f-4b0d-b739-0605cbf935e9-openshift-service-ca\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:07.999723 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:07.999693    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfnth\" (UniqueName: \"kubernetes.io/projected/b00f8ffd-908f-4b0d-b739-0605cbf935e9-kube-api-access-nfnth\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:08.488292 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:08.488252    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00f8ffd-908f-4b0d-b739-0605cbf935e9-maas-api-tls\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:08.490822 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:08.490796    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00f8ffd-908f-4b0d-b739-0605cbf935e9-maas-api-tls\") pod \"maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:08.783380 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:08.783272    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:08.941989 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:08.941951    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"]
Jul 22 15:41:08.946266 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:41:08.946238    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00f8ffd_908f_4b0d_b739_0605cbf935e9.slice/crio-ec029df32d2c5e25d7f142f734b92228e4ff4f6c6fe3001fe6d2c0c7656fae8f WatchSource:0}: Error finding container ec029df32d2c5e25d7f142f734b92228e4ff4f6c6fe3001fe6d2c0c7656fae8f: Status 404 returned error can't find the container with id ec029df32d2c5e25d7f142f734b92228e4ff4f6c6fe3001fe6d2c0c7656fae8f
Jul 22 15:41:09.935146 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:09.935110    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7" event={"ID":"b00f8ffd-908f-4b0d-b739-0605cbf935e9","Type":"ContainerStarted","Data":"ec029df32d2c5e25d7f142f734b92228e4ff4f6c6fe3001fe6d2c0c7656fae8f"}
Jul 22 15:41:10.941152 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:10.941115    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7" event={"ID":"b00f8ffd-908f-4b0d-b739-0605cbf935e9","Type":"ContainerStarted","Data":"a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d"}
Jul 22 15:41:10.941586 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:10.941179    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:10.958592 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:10.958540    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7" podStartSLOduration=2.318332825 podStartE2EDuration="3.958521556s" totalImagesPullingTime="1.640188731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:41:10.957617394 +0000 UTC m=+2209.412331037" watchObservedRunningTime="2026-07-22 15:41:10.958521556 +0000 UTC m=+2209.413235170"
Jul 22 15:41:16.953472 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:16.953417    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:41:20.410043 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.410006    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"]
Jul 22 15:41:20.413962 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.413936    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"
Jul 22 15:41:20.416604 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.416575    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:41:20.421308 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.421277    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"]
Jul 22 15:41:20.503723 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.503686    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065cb4a-f460-4ea9-93de-281b579a4d1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-81c749-vnwsq\" (UID: \"d065cb4a-f460-4ea9-93de-281b579a4d1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"
Jul 22 15:41:20.604789 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.604750    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065cb4a-f460-4ea9-93de-281b579a4d1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-81c749-vnwsq\" (UID: \"d065cb4a-f460-4ea9-93de-281b579a4d1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"
Jul 22 15:41:20.605357 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.605336    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065cb4a-f460-4ea9-93de-281b579a4d1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-81c749-vnwsq\" (UID: \"d065cb4a-f460-4ea9-93de-281b579a4d1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"
Jul 22 15:41:20.726863 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.726773    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"
Jul 22 15:41:20.898141 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.898108    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"]
Jul 22 15:41:20.900203 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:41:20.900177    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd065cb4a_f460_4ea9_93de_281b579a4d1f.slice/crio-da5bd4efb1a9c3d0ac6f425ce8e3b0219382af3b3b3a397354230783743acceb WatchSource:0}: Error finding container da5bd4efb1a9c3d0ac6f425ce8e3b0219382af3b3b3a397354230783743acceb: Status 404 returned error can't find the container with id da5bd4efb1a9c3d0ac6f425ce8e3b0219382af3b3b3a397354230783743acceb
Jul 22 15:41:20.990673 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:20.990579    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq" event={"ID":"d065cb4a-f460-4ea9-93de-281b579a4d1f","Type":"ContainerStarted","Data":"da5bd4efb1a9c3d0ac6f425ce8e3b0219382af3b3b3a397354230783743acceb"}
Jul 22 15:41:21.996971 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:21.996887    2560 generic.go:356] "Generic (PLEG): container finished" podID="d065cb4a-f460-4ea9-93de-281b579a4d1f" containerID="8ea11791ac0726087d8f4e85be61b08cca9be91415910dfcbbfde57c1a70f19d" exitCode=0
Jul 22 15:41:21.997334 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:21.996967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq" event={"ID":"d065cb4a-f460-4ea9-93de-281b579a4d1f","Type":"ContainerDied","Data":"8ea11791ac0726087d8f4e85be61b08cca9be91415910dfcbbfde57c1a70f19d"}
Jul 22 15:41:23.140604 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:23.140575    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"
Jul 22 15:41:23.228908 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:23.228871    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065cb4a-f460-4ea9-93de-281b579a4d1f-maas-api-service-ca\") pod \"d065cb4a-f460-4ea9-93de-281b579a4d1f\" (UID: \"d065cb4a-f460-4ea9-93de-281b579a4d1f\") "
Jul 22 15:41:23.229242 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:23.229214    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d065cb4a-f460-4ea9-93de-281b579a4d1f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d065cb4a-f460-4ea9-93de-281b579a4d1f" (UID: "d065cb4a-f460-4ea9-93de-281b579a4d1f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:41:23.336860 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:23.336754    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d065cb4a-f460-4ea9-93de-281b579a4d1f-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:41:24.008713 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:24.008682    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"
Jul 22 15:41:24.008917 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:24.008716    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq" event={"ID":"d065cb4a-f460-4ea9-93de-281b579a4d1f","Type":"ContainerDied","Data":"da5bd4efb1a9c3d0ac6f425ce8e3b0219382af3b3b3a397354230783743acceb"}
Jul 22 15:41:24.008917 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:24.008760    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da5bd4efb1a9c3d0ac6f425ce8e3b0219382af3b3b3a397354230783743acceb"
Jul 22 15:41:36.296343 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.296309    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"]
Jul 22 15:41:36.296975 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.296585    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg" podUID="4aa6c338-3504-488d-99df-66addde0b82e" containerName="maas-api" containerID="cri-o://9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574" gracePeriod=30
Jul 22 15:41:36.544813 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.544788    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:36.662310 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.662272    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa6c338-3504-488d-99df-66addde0b82e-openshift-service-ca\") pod \"4aa6c338-3504-488d-99df-66addde0b82e\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") "
Jul 22 15:41:36.662557 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.662364    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4jlr\" (UniqueName: \"kubernetes.io/projected/4aa6c338-3504-488d-99df-66addde0b82e-kube-api-access-x4jlr\") pod \"4aa6c338-3504-488d-99df-66addde0b82e\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") "
Jul 22 15:41:36.662557 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.662389    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa6c338-3504-488d-99df-66addde0b82e-maas-api-tls\") pod \"4aa6c338-3504-488d-99df-66addde0b82e\" (UID: \"4aa6c338-3504-488d-99df-66addde0b82e\") "
Jul 22 15:41:36.662762 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.662737    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aa6c338-3504-488d-99df-66addde0b82e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4aa6c338-3504-488d-99df-66addde0b82e" (UID: "4aa6c338-3504-488d-99df-66addde0b82e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:41:36.664607 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.664570    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aa6c338-3504-488d-99df-66addde0b82e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4aa6c338-3504-488d-99df-66addde0b82e" (UID: "4aa6c338-3504-488d-99df-66addde0b82e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:41:36.664607 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.664594    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa6c338-3504-488d-99df-66addde0b82e-kube-api-access-x4jlr" (OuterVolumeSpecName: "kube-api-access-x4jlr") pod "4aa6c338-3504-488d-99df-66addde0b82e" (UID: "4aa6c338-3504-488d-99df-66addde0b82e"). InnerVolumeSpecName "kube-api-access-x4jlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:41:36.763922 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.763880    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa6c338-3504-488d-99df-66addde0b82e-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:41:36.763922 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.763910    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4jlr\" (UniqueName: \"kubernetes.io/projected/4aa6c338-3504-488d-99df-66addde0b82e-kube-api-access-x4jlr\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:41:36.763922 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:36.763923    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4aa6c338-3504-488d-99df-66addde0b82e-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:41:37.075346 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:37.075243    2560 generic.go:356] "Generic (PLEG): container finished" podID="4aa6c338-3504-488d-99df-66addde0b82e" containerID="9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574" exitCode=0
Jul 22 15:41:37.075346 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:37.075292    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"
Jul 22 15:41:37.075346 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:37.075333    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg" event={"ID":"4aa6c338-3504-488d-99df-66addde0b82e","Type":"ContainerDied","Data":"9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574"}
Jul 22 15:41:37.075611 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:37.075390    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg" event={"ID":"4aa6c338-3504-488d-99df-66addde0b82e","Type":"ContainerDied","Data":"682beb1883834b3af4a47925c503c6086e174096d6da4ae5916402858f395be3"}
Jul 22 15:41:37.075611 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:37.075412    2560 scope.go:117] "RemoveContainer" containerID="9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574"
Jul 22 15:41:37.085184 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:37.085159    2560 scope.go:117] "RemoveContainer" containerID="9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574"
Jul 22 15:41:37.085506 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:41:37.085485    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574\": container with ID starting with 9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574 not found: ID does not exist" containerID="9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574"
Jul 22 15:41:37.085571 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:37.085515    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574"} err="failed to get container status \"9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574\": rpc error: code = NotFound desc = could not find container \"9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574\": container with ID starting with 9af5e850c792d931dc7f6ced6ca28b56a06a85433926a44dbe1ccf7833f96574 not found: ID does not exist"
Jul 22 15:41:37.099201 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:37.099166    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"]
Jul 22 15:41:37.103131 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:37.103106    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-81c749-69b96dc9c-z88zg"]
Jul 22 15:41:38.601678 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:38.601645    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aa6c338-3504-488d-99df-66addde0b82e" path="/var/lib/kubelet/pods/4aa6c338-3504-488d-99df-66addde0b82e/volumes"
Jul 22 15:41:40.515079 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:40.515040    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"]
Jul 22 15:41:40.520443 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:40.520414    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-81c749-vnwsq"]
Jul 22 15:41:40.602364 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:40.602328    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d065cb4a-f460-4ea9-93de-281b579a4d1f" path="/var/lib/kubelet/pods/d065cb4a-f460-4ea9-93de-281b579a4d1f/volumes"
Jul 22 15:41:47.607834 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.607743    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"]
Jul 22 15:41:47.608247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.608141    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d065cb4a-f460-4ea9-93de-281b579a4d1f" containerName="revoke-keys"
Jul 22 15:41:47.608247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.608154    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d065cb4a-f460-4ea9-93de-281b579a4d1f" containerName="revoke-keys"
Jul 22 15:41:47.608247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.608169    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aa6c338-3504-488d-99df-66addde0b82e" containerName="maas-api"
Jul 22 15:41:47.608247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.608174    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aa6c338-3504-488d-99df-66addde0b82e" containerName="maas-api"
Jul 22 15:41:47.608247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.608247    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aa6c338-3504-488d-99df-66addde0b82e" containerName="maas-api"
Jul 22 15:41:47.608436 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.608255    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d065cb4a-f460-4ea9-93de-281b579a4d1f" containerName="revoke-keys"
Jul 22 15:41:47.612856 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.612838    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"
Jul 22 15:41:47.615549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.615525    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:41:47.619584 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.619552    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"]
Jul 22 15:41:47.650967 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.650929    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f0d539-310e-4dd8-97b5-5c3540459df4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc\" (UID: \"95f0d539-310e-4dd8-97b5-5c3540459df4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"
Jul 22 15:41:47.752014 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.751976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f0d539-310e-4dd8-97b5-5c3540459df4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc\" (UID: \"95f0d539-310e-4dd8-97b5-5c3540459df4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"
Jul 22 15:41:47.752621 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.752599    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f0d539-310e-4dd8-97b5-5c3540459df4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc\" (UID: \"95f0d539-310e-4dd8-97b5-5c3540459df4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"
Jul 22 15:41:47.925769 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:47.925741    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"
Jul 22 15:41:48.058512 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:48.057251    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"]
Jul 22 15:41:48.132831 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:48.132796    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc" event={"ID":"95f0d539-310e-4dd8-97b5-5c3540459df4","Type":"ContainerStarted","Data":"f3cc2020a1ff5ad41dae40451861d439ff66feb0d3303d79e2f74e0b5c9ec454"}
Jul 22 15:41:49.138936 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:49.138842    2560 generic.go:356] "Generic (PLEG): container finished" podID="95f0d539-310e-4dd8-97b5-5c3540459df4" containerID="6fe7a4a3e8e8316aed9eea0d174cc506be40b05324256fe71887860df651ee46" exitCode=0
Jul 22 15:41:49.139333 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:49.138930    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc" event={"ID":"95f0d539-310e-4dd8-97b5-5c3540459df4","Type":"ContainerDied","Data":"6fe7a4a3e8e8316aed9eea0d174cc506be40b05324256fe71887860df651ee46"}
Jul 22 15:41:50.277730 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:50.277703    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"
Jul 22 15:41:50.374230 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:50.374188    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f0d539-310e-4dd8-97b5-5c3540459df4-maas-api-service-ca\") pod \"95f0d539-310e-4dd8-97b5-5c3540459df4\" (UID: \"95f0d539-310e-4dd8-97b5-5c3540459df4\") "
Jul 22 15:41:50.374585 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:50.374562    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95f0d539-310e-4dd8-97b5-5c3540459df4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95f0d539-310e-4dd8-97b5-5c3540459df4" (UID: "95f0d539-310e-4dd8-97b5-5c3540459df4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:41:50.475776 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:50.475682    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95f0d539-310e-4dd8-97b5-5c3540459df4-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:41:51.149547 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:51.149515    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"
Jul 22 15:41:51.149547 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:51.149531    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc" event={"ID":"95f0d539-310e-4dd8-97b5-5c3540459df4","Type":"ContainerDied","Data":"f3cc2020a1ff5ad41dae40451861d439ff66feb0d3303d79e2f74e0b5c9ec454"}
Jul 22 15:41:51.149809 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:41:51.149581    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3cc2020a1ff5ad41dae40451861d439ff66feb0d3303d79e2f74e0b5c9ec454"
Jul 22 15:42:03.108721 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.108684    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"]
Jul 22 15:42:03.109520 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.109483    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7" podUID="b00f8ffd-908f-4b0d-b739-0605cbf935e9" containerName="maas-api" containerID="cri-o://a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d" gracePeriod=30
Jul 22 15:42:03.363241 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.363175    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:42:03.490953 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.490911    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00f8ffd-908f-4b0d-b739-0605cbf935e9-maas-api-tls\") pod \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") "
Jul 22 15:42:03.490953 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.490958    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfnth\" (UniqueName: \"kubernetes.io/projected/b00f8ffd-908f-4b0d-b739-0605cbf935e9-kube-api-access-nfnth\") pod \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") "
Jul 22 15:42:03.491171 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.491011    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00f8ffd-908f-4b0d-b739-0605cbf935e9-openshift-service-ca\") pod \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\" (UID: \"b00f8ffd-908f-4b0d-b739-0605cbf935e9\") "
Jul 22 15:42:03.491497 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.491433    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00f8ffd-908f-4b0d-b739-0605cbf935e9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b00f8ffd-908f-4b0d-b739-0605cbf935e9" (UID: "b00f8ffd-908f-4b0d-b739-0605cbf935e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:42:03.493119 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.493091    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b00f8ffd-908f-4b0d-b739-0605cbf935e9-kube-api-access-nfnth" (OuterVolumeSpecName: "kube-api-access-nfnth") pod "b00f8ffd-908f-4b0d-b739-0605cbf935e9" (UID: "b00f8ffd-908f-4b0d-b739-0605cbf935e9"). InnerVolumeSpecName "kube-api-access-nfnth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:42:03.493198 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.493160    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b00f8ffd-908f-4b0d-b739-0605cbf935e9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b00f8ffd-908f-4b0d-b739-0605cbf935e9" (UID: "b00f8ffd-908f-4b0d-b739-0605cbf935e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:42:03.592758 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.592718    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00f8ffd-908f-4b0d-b739-0605cbf935e9-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:42:03.592758 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.592753    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfnth\" (UniqueName: \"kubernetes.io/projected/b00f8ffd-908f-4b0d-b739-0605cbf935e9-kube-api-access-nfnth\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:42:03.592758 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:03.592764    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00f8ffd-908f-4b0d-b739-0605cbf935e9-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:42:04.214467 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.214421    2560 generic.go:356] "Generic (PLEG): container finished" podID="b00f8ffd-908f-4b0d-b739-0605cbf935e9" containerID="a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d" exitCode=0
Jul 22 15:42:04.214905 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.214499    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"
Jul 22 15:42:04.214905 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.214512    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7" event={"ID":"b00f8ffd-908f-4b0d-b739-0605cbf935e9","Type":"ContainerDied","Data":"a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d"}
Jul 22 15:42:04.214905 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.214568    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7" event={"ID":"b00f8ffd-908f-4b0d-b739-0605cbf935e9","Type":"ContainerDied","Data":"ec029df32d2c5e25d7f142f734b92228e4ff4f6c6fe3001fe6d2c0c7656fae8f"}
Jul 22 15:42:04.214905 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.214590    2560 scope.go:117] "RemoveContainer" containerID="a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d"
Jul 22 15:42:04.223541 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.223522    2560 scope.go:117] "RemoveContainer" containerID="a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d"
Jul 22 15:42:04.223813 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:42:04.223794    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d\": container with ID starting with a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d not found: ID does not exist" containerID="a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d"
Jul 22 15:42:04.223857 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.223821    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d"} err="failed to get container status \"a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d\": rpc error: code = NotFound desc = could not find container \"a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d\": container with ID starting with a3d1a267f8c448a29142a274b81f47ea9e4482a13c949f50ff77b970fe2f4d9d not found: ID does not exist"
Jul 22 15:42:04.236934 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.236908    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"]
Jul 22 15:42:04.242788 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.242762    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b47c9e-5cdd569c7d-h8jn7"]
Jul 22 15:42:04.602505 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:04.602407    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b00f8ffd-908f-4b0d-b739-0605cbf935e9" path="/var/lib/kubelet/pods/b00f8ffd-908f-4b0d-b739-0605cbf935e9/volumes"
Jul 22 15:42:07.700611 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:07.700579    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"]
Jul 22 15:42:07.703385 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:07.703357    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b47c9e-fpgsc"]
Jul 22 15:42:08.602522 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:08.602489    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95f0d539-310e-4dd8-97b5-5c3540459df4" path="/var/lib/kubelet/pods/95f0d539-310e-4dd8-97b5-5c3540459df4/volumes"
Jul 22 15:42:18.061960 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.061916    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"]
Jul 22 15:42:18.062422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.062347    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95f0d539-310e-4dd8-97b5-5c3540459df4" containerName="revoke-keys"
Jul 22 15:42:18.062422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.062361    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="95f0d539-310e-4dd8-97b5-5c3540459df4" containerName="revoke-keys"
Jul 22 15:42:18.062422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.062373    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b00f8ffd-908f-4b0d-b739-0605cbf935e9" containerName="maas-api"
Jul 22 15:42:18.062422 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.062378    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b00f8ffd-908f-4b0d-b739-0605cbf935e9" containerName="maas-api"
Jul 22 15:42:18.062712 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.062471    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="95f0d539-310e-4dd8-97b5-5c3540459df4" containerName="revoke-keys"
Jul 22 15:42:18.062712 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.062485    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b00f8ffd-908f-4b0d-b739-0605cbf935e9" containerName="maas-api"
Jul 22 15:42:18.066993 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.066972    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.069959 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.069932    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-6f1c00-serving-cert\""
Jul 22 15:42:18.069959 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.069953    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8q48q\""
Jul 22 15:42:18.076245 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.076218    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"]
Jul 22 15:42:18.228833 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.228788    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t8s8\" (UniqueName: \"kubernetes.io/projected/d2cd472f-17a8-4642-8760-4df286ef822b-kube-api-access-6t8s8\") pod \"maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.229035 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.228849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd472f-17a8-4642-8760-4df286ef822b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.229035 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.228936    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd472f-17a8-4642-8760-4df286ef822b-maas-api-tls\") pod \"maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.329763 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.329661    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t8s8\" (UniqueName: \"kubernetes.io/projected/d2cd472f-17a8-4642-8760-4df286ef822b-kube-api-access-6t8s8\") pod \"maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.329763 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.329710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd472f-17a8-4642-8760-4df286ef822b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.329763 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.329758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd472f-17a8-4642-8760-4df286ef822b-maas-api-tls\") pod \"maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.330387 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.330365    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd472f-17a8-4642-8760-4df286ef822b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.332320 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.332303    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd472f-17a8-4642-8760-4df286ef822b-maas-api-tls\") pod \"maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.338651 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.338626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t8s8\" (UniqueName: \"kubernetes.io/projected/d2cd472f-17a8-4642-8760-4df286ef822b-kube-api-access-6t8s8\") pod \"maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.378894 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.378855    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:18.516375 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:18.516350    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"]
Jul 22 15:42:18.518575 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:42:18.518529    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2cd472f_17a8_4642_8760_4df286ef822b.slice/crio-6c755a5a3a81858007adde9ef045d00a2228b05c7dfd5e987ea9732383bf1677 WatchSource:0}: Error finding container 6c755a5a3a81858007adde9ef045d00a2228b05c7dfd5e987ea9732383bf1677: Status 404 returned error can't find the container with id 6c755a5a3a81858007adde9ef045d00a2228b05c7dfd5e987ea9732383bf1677
Jul 22 15:42:19.290004 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:19.289968    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t" event={"ID":"d2cd472f-17a8-4642-8760-4df286ef822b","Type":"ContainerStarted","Data":"46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118"}
Jul 22 15:42:19.290410 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:19.290025    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t" event={"ID":"d2cd472f-17a8-4642-8760-4df286ef822b","Type":"ContainerStarted","Data":"6c755a5a3a81858007adde9ef045d00a2228b05c7dfd5e987ea9732383bf1677"}
Jul 22 15:42:19.290410 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:19.290069    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:25.301207 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:25.301175    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:42:25.318520 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:25.318428    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t" podStartSLOduration=6.955106772 podStartE2EDuration="7.31841265s" totalImagesPullingTime="363.305878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:42:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:42:19.325886926 +0000 UTC m=+2277.780600542" watchObservedRunningTime="2026-07-22 15:42:25.31841265 +0000 UTC m=+2283.773126264"
Jul 22 15:42:26.110673 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.110637    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"]
Jul 22 15:42:26.116012 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.115987    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.119314 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.119284    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-552a8b-serving-cert\""
Jul 22 15:42:26.127536 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.127499    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"]
Jul 22 15:42:26.205724 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.205690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6ce32ef-d253-4998-b4af-249828380bf3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.205907 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.205755    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.205907 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.205815    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f5f5\" (UniqueName: \"kubernetes.io/projected/f6ce32ef-d253-4998-b4af-249828380bf3-kube-api-access-9f5f5\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.306642 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.306600    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6ce32ef-d253-4998-b4af-249828380bf3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.307036 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.306658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.307036 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.306704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f5f5\" (UniqueName: \"kubernetes.io/projected/f6ce32ef-d253-4998-b4af-249828380bf3-kube-api-access-9f5f5\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.307036 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:42:26.306841    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-serving-cert: secret "maas-api-e2e-shared-b-552a8b-serving-cert" not found
Jul 22 15:42:26.307036 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:42:26.306916    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls podName:f6ce32ef-d253-4998-b4af-249828380bf3 nodeName:}" failed. No retries permitted until 2026-07-22 15:42:26.806895473 +0000 UTC m=+2285.261609070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls") pod "maas-api-e2e-shared-b-552a8b-654899579d-5qbtl" (UID: "f6ce32ef-d253-4998-b4af-249828380bf3") : secret "maas-api-e2e-shared-b-552a8b-serving-cert" not found
Jul 22 15:42:26.307274 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.307254    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6ce32ef-d253-4998-b4af-249828380bf3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.318180 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.318151    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f5f5\" (UniqueName: \"kubernetes.io/projected/f6ce32ef-d253-4998-b4af-249828380bf3-kube-api-access-9f5f5\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.811233 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:26.811173    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:26.811434 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:42:26.811360    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-serving-cert: secret "maas-api-e2e-shared-b-552a8b-serving-cert" not found
Jul 22 15:42:26.811498 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:42:26.811471    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls podName:f6ce32ef-d253-4998-b4af-249828380bf3 nodeName:}" failed. No retries permitted until 2026-07-22 15:42:27.811436349 +0000 UTC m=+2286.266149957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls") pod "maas-api-e2e-shared-b-552a8b-654899579d-5qbtl" (UID: "f6ce32ef-d253-4998-b4af-249828380bf3") : secret "maas-api-e2e-shared-b-552a8b-serving-cert" not found
Jul 22 15:42:27.822213 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:27.822173    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:27.822680 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:42:27.822329    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-serving-cert: secret "maas-api-e2e-shared-b-552a8b-serving-cert" not found
Jul 22 15:42:27.822680 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:42:27.822396    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls podName:f6ce32ef-d253-4998-b4af-249828380bf3 nodeName:}" failed. No retries permitted until 2026-07-22 15:42:29.82237958 +0000 UTC m=+2288.277093173 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls") pod "maas-api-e2e-shared-b-552a8b-654899579d-5qbtl" (UID: "f6ce32ef-d253-4998-b4af-249828380bf3") : secret "maas-api-e2e-shared-b-552a8b-serving-cert" not found
Jul 22 15:42:29.842460 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:29.842418    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:29.845090 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:29.845063    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls\") pod \"maas-api-e2e-shared-b-552a8b-654899579d-5qbtl\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:30.029227 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:30.029188    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:30.165546 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:30.165520    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"]
Jul 22 15:42:30.167947 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:42:30.167907    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ce32ef_d253_4998_b4af_249828380bf3.slice/crio-e22aa014f26e656e2e5a7021062c3cb3ae84887255eaea006f282ef22080fb46 WatchSource:0}: Error finding container e22aa014f26e656e2e5a7021062c3cb3ae84887255eaea006f282ef22080fb46: Status 404 returned error can't find the container with id e22aa014f26e656e2e5a7021062c3cb3ae84887255eaea006f282ef22080fb46
Jul 22 15:42:30.349340 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:30.349299    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl" event={"ID":"f6ce32ef-d253-4998-b4af-249828380bf3","Type":"ContainerStarted","Data":"e22aa014f26e656e2e5a7021062c3cb3ae84887255eaea006f282ef22080fb46"}
Jul 22 15:42:31.357090 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:31.357056    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl" event={"ID":"f6ce32ef-d253-4998-b4af-249828380bf3","Type":"ContainerStarted","Data":"6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d"}
Jul 22 15:42:31.357495 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:31.357117    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:31.377160 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:31.377095    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl" podStartSLOduration=4.975476892 podStartE2EDuration="5.377076285s" totalImagesPullingTime="401.599393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:42:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:42:31.376620547 +0000 UTC m=+2289.831334163" watchObservedRunningTime="2026-07-22 15:42:31.377076285 +0000 UTC m=+2289.831789900"
Jul 22 15:42:37.367974 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:37.367938    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:42:44.097605 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.097571    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"]
Jul 22 15:42:44.101610 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.101589    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.104929 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.104901    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6f1c00\"/\"openshift-service-ca.crt\""
Jul 22 15:42:44.104929 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.104901    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6f1c00\"/\"kube-root-ca.crt\""
Jul 22 15:42:44.105998 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.105973    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6f1c00\"/\"auth-test-model-6f1c00-kserve-self-signed-certs\""
Jul 22 15:42:44.106172 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.106128    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6f1c00\"/\"default-dockercfg-pfltb\""
Jul 22 15:42:44.111426 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.111404    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"]
Jul 22 15:42:44.148534 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.148501    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-tmp-dir\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.148704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.148541    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-model-cache\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.148704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.148561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58be4fc6-222c-4789-be65-df1485f1e172-tls-certs\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.148704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.148591    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjjfg\" (UniqueName: \"kubernetes.io/projected/58be4fc6-222c-4789-be65-df1485f1e172-kube-api-access-zjjfg\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.148704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.148634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-home\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.148704 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.148660    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-dshm\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.249149 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.249111    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-tmp-dir\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.249149 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.249154    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-model-cache\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.249413 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.249175    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58be4fc6-222c-4789-be65-df1485f1e172-tls-certs\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.249413 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.249310    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjjfg\" (UniqueName: \"kubernetes.io/projected/58be4fc6-222c-4789-be65-df1485f1e172-kube-api-access-zjjfg\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.249528 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.249413    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-home\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.249528 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.249503    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-dshm\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.249641 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.249555    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-tmp-dir\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.249641 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.249634    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-model-cache\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.250004 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.249976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-home\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.251580 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.251558    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-dshm\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.251785 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.251769    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58be4fc6-222c-4789-be65-df1485f1e172-tls-certs\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.260993 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.260966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjjfg\" (UniqueName: \"kubernetes.io/projected/58be4fc6-222c-4789-be65-df1485f1e172-kube-api-access-zjjfg\") pod \"auth-test-model-6f1c00-kserve-7854b6f999-rm82l\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") " pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.411898 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.411853    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:44.555184 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:44.555146    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"]
Jul 22 15:42:44.558771 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:42:44.558736    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58be4fc6_222c_4789_be65_df1485f1e172.slice/crio-cf857a0ba8bf7cb7a48f6a4ec59fe3b60e9e56b843b423451fc8b89724d6817e WatchSource:0}: Error finding container cf857a0ba8bf7cb7a48f6a4ec59fe3b60e9e56b843b423451fc8b89724d6817e: Status 404 returned error can't find the container with id cf857a0ba8bf7cb7a48f6a4ec59fe3b60e9e56b843b423451fc8b89724d6817e
Jul 22 15:42:45.429747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:45.429711    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l" event={"ID":"58be4fc6-222c-4789-be65-df1485f1e172","Type":"ContainerStarted","Data":"66f8181b9468722b40537d6597f0f1084a2fbf4789cea030c4312e95e286a434"}
Jul 22 15:42:45.430174 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:45.429757    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l" event={"ID":"58be4fc6-222c-4789-be65-df1485f1e172","Type":"ContainerStarted","Data":"cf857a0ba8bf7cb7a48f6a4ec59fe3b60e9e56b843b423451fc8b89724d6817e"}
Jul 22 15:42:45.452178 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:45.452113    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l" podStartSLOduration=1.452096167 podStartE2EDuration="1.452096167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:42:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:42:45.44961186 +0000 UTC m=+2303.904325499" watchObservedRunningTime="2026-07-22 15:42:45.452096167 +0000 UTC m=+2303.906809781"
Jul 22 15:42:54.412486 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:54.412424    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:54.412995 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:54.412507    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:54.425180 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:54.425147    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:42:54.484093 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:42:54.484063    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:44:22.210834 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:44:22.210726    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:44:22.217811 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:44:22.217784    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:44:22.228276 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:44:22.228256    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:44:22.235036 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:44:22.235016    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:44:55.001187 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:44:55.001113    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"]
Jul 22 15:45:00.182056 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.182022    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x"]
Jul 22 15:45:00.185490 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.185469    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k"]
Jul 22 15:45:00.185618 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.185576    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x"
Jul 22 15:45:00.188718 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.188699    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v"]
Jul 22 15:45:00.188838 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.188823    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k"
Jul 22 15:45:00.189683 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.189667    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:45:00.191853 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.191837    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v"
Jul 22 15:45:00.195804 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.195783    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k"]
Jul 22 15:45:00.197595 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.197579    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x"
Jul 22 15:45:00.205990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.205966    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x"]
Jul 22 15:45:00.209426 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.208199    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v"]
Jul 22 15:45:00.211629 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.211607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k"
Jul 22 15:45:00.218418 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.218399    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v"
Jul 22 15:45:00.372590 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.372564    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x"]
Jul 22 15:45:00.374578 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:45:00.374550    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6fc25b_587e_4465_a010_516bb4172c5e.slice/crio-778518e2b66096b152b115a2ca400503a6e2778e6d6052eea8929884213b02a5 WatchSource:0}: Error finding container 778518e2b66096b152b115a2ca400503a6e2778e6d6052eea8929884213b02a5: Status 404 returned error can't find the container with id 778518e2b66096b152b115a2ca400503a6e2778e6d6052eea8929884213b02a5
Jul 22 15:45:00.419557 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.419535    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v"]
Jul 22 15:45:00.421240 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:45:00.421215    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c4de1a6_cb19_42cd_a64b_3e76b3274d2d.slice/crio-db42f0d3c1e2cfecb8fac27dafea8cb4385f045f65b61ff155151a9539291580 WatchSource:0}: Error finding container db42f0d3c1e2cfecb8fac27dafea8cb4385f045f65b61ff155151a9539291580: Status 404 returned error can't find the container with id db42f0d3c1e2cfecb8fac27dafea8cb4385f045f65b61ff155151a9539291580
Jul 22 15:45:00.595895 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:00.595865    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k"]
Jul 22 15:45:00.598073 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:45:00.598042    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e084579_dc9a_4a5e_9164_dd96daea2b59.slice/crio-2a96a34c6f01fea589453c2849e5e238ab366e04a7d6080f173ae7f0c63b8c8f WatchSource:0}: Error finding container 2a96a34c6f01fea589453c2849e5e238ab366e04a7d6080f173ae7f0c63b8c8f: Status 404 returned error can't find the container with id 2a96a34c6f01fea589453c2849e5e238ab366e04a7d6080f173ae7f0c63b8c8f
Jul 22 15:45:01.105099 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:01.105001    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k" event={"ID":"2e084579-dc9a-4a5e-9164-dd96daea2b59","Type":"ContainerStarted","Data":"b2a30d823c558b4c562ff91fc2b984839187cc2abde4e0ea5fbe16b65576728a"}
Jul 22 15:45:01.105099 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:01.105068    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k" event={"ID":"2e084579-dc9a-4a5e-9164-dd96daea2b59","Type":"ContainerStarted","Data":"2a96a34c6f01fea589453c2849e5e238ab366e04a7d6080f173ae7f0c63b8c8f"}
Jul 22 15:45:01.106513 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:01.106484    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v" event={"ID":"5c4de1a6-cb19-42cd-a64b-3e76b3274d2d","Type":"ContainerStarted","Data":"9f92fe9e5f0d00cb8c0074961f9cb927aec97028931febf1424650a7ff987cd9"}
Jul 22 15:45:01.106669 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:01.106534    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v" event={"ID":"5c4de1a6-cb19-42cd-a64b-3e76b3274d2d","Type":"ContainerStarted","Data":"db42f0d3c1e2cfecb8fac27dafea8cb4385f045f65b61ff155151a9539291580"}
Jul 22 15:45:01.107936 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:01.107916    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x" event={"ID":"af6fc25b-587e-4465-a010-516bb4172c5e","Type":"ContainerStarted","Data":"f50fa6d4e59583ec8ebf9505903d17fce448026b7551d991b767d47fd0a06311"}
Jul 22 15:45:01.108022 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:01.107954    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x" event={"ID":"af6fc25b-587e-4465-a010-516bb4172c5e","Type":"ContainerStarted","Data":"778518e2b66096b152b115a2ca400503a6e2778e6d6052eea8929884213b02a5"}
Jul 22 15:45:01.121046 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:01.121004    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k" podStartSLOduration=1.120992285 podStartE2EDuration="1.120992285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:45:01.120679663 +0000 UTC m=+2439.575393279" watchObservedRunningTime="2026-07-22 15:45:01.120992285 +0000 UTC m=+2439.575705878"
Jul 22 15:45:01.135053 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:01.134941    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v" podStartSLOduration=1.134927364 podStartE2EDuration="1.134927364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:45:01.134256968 +0000 UTC m=+2439.588970584" watchObservedRunningTime="2026-07-22 15:45:01.134927364 +0000 UTC m=+2439.589640981"
Jul 22 15:45:01.149365 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:01.149330    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x" podStartSLOduration=1.14931891 podStartE2EDuration="1.14931891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:45:01.148219471 +0000 UTC m=+2439.602933086" watchObservedRunningTime="2026-07-22 15:45:01.14931891 +0000 UTC m=+2439.604032524"
Jul 22 15:45:02.113959 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:02.113918    2560 generic.go:356] "Generic (PLEG): container finished" podID="5c4de1a6-cb19-42cd-a64b-3e76b3274d2d" containerID="9f92fe9e5f0d00cb8c0074961f9cb927aec97028931febf1424650a7ff987cd9" exitCode=0
Jul 22 15:45:02.114408 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:02.114004    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v" event={"ID":"5c4de1a6-cb19-42cd-a64b-3e76b3274d2d","Type":"ContainerDied","Data":"9f92fe9e5f0d00cb8c0074961f9cb927aec97028931febf1424650a7ff987cd9"}
Jul 22 15:45:02.115591 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:02.115569    2560 generic.go:356] "Generic (PLEG): container finished" podID="af6fc25b-587e-4465-a010-516bb4172c5e" containerID="f50fa6d4e59583ec8ebf9505903d17fce448026b7551d991b767d47fd0a06311" exitCode=0
Jul 22 15:45:02.115732 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:02.115658    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x" event={"ID":"af6fc25b-587e-4465-a010-516bb4172c5e","Type":"ContainerDied","Data":"f50fa6d4e59583ec8ebf9505903d17fce448026b7551d991b767d47fd0a06311"}
Jul 22 15:45:02.117060 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:02.117038    2560 generic.go:356] "Generic (PLEG): container finished" podID="2e084579-dc9a-4a5e-9164-dd96daea2b59" containerID="b2a30d823c558b4c562ff91fc2b984839187cc2abde4e0ea5fbe16b65576728a" exitCode=0
Jul 22 15:45:02.117170 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:02.117084    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k" event={"ID":"2e084579-dc9a-4a5e-9164-dd96daea2b59","Type":"ContainerDied","Data":"b2a30d823c558b4c562ff91fc2b984839187cc2abde4e0ea5fbe16b65576728a"}
Jul 22 15:45:03.284696 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.284672    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k"
Jul 22 15:45:03.288474 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.288440    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x"
Jul 22 15:45:03.322583 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.322553    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v"
Jul 22 15:45:03.880084 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880053    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"]
Jul 22 15:45:03.880437 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880421    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af6fc25b-587e-4465-a010-516bb4172c5e" containerName="cleanup"
Jul 22 15:45:03.880437 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880436    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="af6fc25b-587e-4465-a010-516bb4172c5e" containerName="cleanup"
Jul 22 15:45:03.880630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880485    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c4de1a6-cb19-42cd-a64b-3e76b3274d2d" containerName="cleanup"
Jul 22 15:45:03.880630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880492    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c4de1a6-cb19-42cd-a64b-3e76b3274d2d" containerName="cleanup"
Jul 22 15:45:03.880630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880508    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e084579-dc9a-4a5e-9164-dd96daea2b59" containerName="cleanup"
Jul 22 15:45:03.880630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880513    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e084579-dc9a-4a5e-9164-dd96daea2b59" containerName="cleanup"
Jul 22 15:45:03.880630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880572    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c4de1a6-cb19-42cd-a64b-3e76b3274d2d" containerName="cleanup"
Jul 22 15:45:03.880630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880582    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="af6fc25b-587e-4465-a010-516bb4172c5e" containerName="cleanup"
Jul 22 15:45:03.880630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.880589    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e084579-dc9a-4a5e-9164-dd96daea2b59" containerName="cleanup"
Jul 22 15:45:03.883467 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.883439    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:03.887097 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.887073    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6f1c00\"/\"sub-test-model-6f1c00-kserve-self-signed-certs\""
Jul 22 15:45:03.896751 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.896734    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"]
Jul 22 15:45:03.951126 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.951094    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrqtg\" (UniqueName: \"kubernetes.io/projected/94af53ff-12f0-4200-a415-3c962b7cc1fa-kube-api-access-nrqtg\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:03.951244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.951134    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-home\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:03.951244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.951169    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-model-cache\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:03.951244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.951194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-tmp-dir\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:03.951244 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.951239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94af53ff-12f0-4200-a415-3c962b7cc1fa-tls-certs\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:03.951473 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:03.951324    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-dshm\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.051969 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.051931    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-dshm\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.052142 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.052005    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrqtg\" (UniqueName: \"kubernetes.io/projected/94af53ff-12f0-4200-a415-3c962b7cc1fa-kube-api-access-nrqtg\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.052142 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.052033    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-home\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.052142 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.052064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-model-cache\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.052142 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.052089    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-tmp-dir\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.052142 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.052119    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94af53ff-12f0-4200-a415-3c962b7cc1fa-tls-certs\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.052499 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.052475    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-home\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.052578 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.052475    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-model-cache\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.052578 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.052541    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-tmp-dir\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.054192 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.054173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-dshm\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.054497 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.054478    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94af53ff-12f0-4200-a415-3c962b7cc1fa-tls-certs\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.061378 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.061356    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrqtg\" (UniqueName: \"kubernetes.io/projected/94af53ff-12f0-4200-a415-3c962b7cc1fa-kube-api-access-nrqtg\") pod \"sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") " pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.128595 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.128575    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v"
Jul 22 15:45:04.128595 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.128581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v" event={"ID":"5c4de1a6-cb19-42cd-a64b-3e76b3274d2d","Type":"ContainerDied","Data":"db42f0d3c1e2cfecb8fac27dafea8cb4385f045f65b61ff155151a9539291580"}
Jul 22 15:45:04.128793 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.128626    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db42f0d3c1e2cfecb8fac27dafea8cb4385f045f65b61ff155151a9539291580"
Jul 22 15:45:04.129982 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.129962    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x"
Jul 22 15:45:04.130124 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.129983    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745585-p989x" event={"ID":"af6fc25b-587e-4465-a010-516bb4172c5e","Type":"ContainerDied","Data":"778518e2b66096b152b115a2ca400503a6e2778e6d6052eea8929884213b02a5"}
Jul 22 15:45:04.130124 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.130025    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="778518e2b66096b152b115a2ca400503a6e2778e6d6052eea8929884213b02a5"
Jul 22 15:45:04.138873 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.138255    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k" event={"ID":"2e084579-dc9a-4a5e-9164-dd96daea2b59","Type":"ContainerDied","Data":"2a96a34c6f01fea589453c2849e5e238ab366e04a7d6080f173ae7f0c63b8c8f"}
Jul 22 15:45:04.138873 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.138283    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k"
Jul 22 15:45:04.138873 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.138347    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a96a34c6f01fea589453c2849e5e238ab366e04a7d6080f173ae7f0c63b8c8f"
Jul 22 15:45:04.193296 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.193277    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:04.328575 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:04.328548    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"]
Jul 22 15:45:04.330284 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:45:04.330253    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94af53ff_12f0_4200_a415_3c962b7cc1fa.slice/crio-00a11e44aaf902e5d304a0a6d09a4e0e8c3198db8b6e7f0e828259dbcddd23f1 WatchSource:0}: Error finding container 00a11e44aaf902e5d304a0a6d09a4e0e8c3198db8b6e7f0e828259dbcddd23f1: Status 404 returned error can't find the container with id 00a11e44aaf902e5d304a0a6d09a4e0e8c3198db8b6e7f0e828259dbcddd23f1
Jul 22 15:45:05.145167 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:05.145134    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb" event={"ID":"94af53ff-12f0-4200-a415-3c962b7cc1fa","Type":"ContainerStarted","Data":"eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe"}
Jul 22 15:45:05.145334 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:05.145178    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb" event={"ID":"94af53ff-12f0-4200-a415-3c962b7cc1fa","Type":"ContainerStarted","Data":"00a11e44aaf902e5d304a0a6d09a4e0e8c3198db8b6e7f0e828259dbcddd23f1"}
Jul 22 15:45:05.165337 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:05.165286    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb" podStartSLOduration=2.165271423 podStartE2EDuration="2.165271423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:45:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:45:05.163397657 +0000 UTC m=+2443.618111273" watchObservedRunningTime="2026-07-22 15:45:05.165271423 +0000 UTC m=+2443.619985037"
Jul 22 15:45:10.020826 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.020764    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l" podUID="58be4fc6-222c-4789-be65-df1485f1e172" containerName="main" containerID="cri-o://66f8181b9468722b40537d6597f0f1084a2fbf4789cea030c4312e95e286a434" gracePeriod=45
Jul 22 15:45:10.172930 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.172894    2560 generic.go:356] "Generic (PLEG): container finished" podID="58be4fc6-222c-4789-be65-df1485f1e172" containerID="66f8181b9468722b40537d6597f0f1084a2fbf4789cea030c4312e95e286a434" exitCode=0
Jul 22 15:45:10.173095 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.172967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l" event={"ID":"58be4fc6-222c-4789-be65-df1485f1e172","Type":"ContainerDied","Data":"66f8181b9468722b40537d6597f0f1084a2fbf4789cea030c4312e95e286a434"}
Jul 22 15:45:10.275178 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.275109    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:45:10.309112 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309085    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjjfg\" (UniqueName: \"kubernetes.io/projected/58be4fc6-222c-4789-be65-df1485f1e172-kube-api-access-zjjfg\") pod \"58be4fc6-222c-4789-be65-df1485f1e172\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") "
Jul 22 15:45:10.309274 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309137    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58be4fc6-222c-4789-be65-df1485f1e172-tls-certs\") pod \"58be4fc6-222c-4789-be65-df1485f1e172\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") "
Jul 22 15:45:10.309274 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309199    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-model-cache\") pod \"58be4fc6-222c-4789-be65-df1485f1e172\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") "
Jul 22 15:45:10.309274 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309225    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-dshm\") pod \"58be4fc6-222c-4789-be65-df1485f1e172\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") "
Jul 22 15:45:10.309274 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309253    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-home\") pod \"58be4fc6-222c-4789-be65-df1485f1e172\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") "
Jul 22 15:45:10.309515 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309329    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-tmp-dir\") pod \"58be4fc6-222c-4789-be65-df1485f1e172\" (UID: \"58be4fc6-222c-4789-be65-df1485f1e172\") "
Jul 22 15:45:10.309515 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309492    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-model-cache" (OuterVolumeSpecName: "model-cache") pod "58be4fc6-222c-4789-be65-df1485f1e172" (UID: "58be4fc6-222c-4789-be65-df1485f1e172"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:45:10.309631 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309586    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-home" (OuterVolumeSpecName: "home") pod "58be4fc6-222c-4789-be65-df1485f1e172" (UID: "58be4fc6-222c-4789-be65-df1485f1e172"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:45:10.309734 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309710    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:45:10.309808 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309739    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:45:10.309808 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.309735    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "58be4fc6-222c-4789-be65-df1485f1e172" (UID: "58be4fc6-222c-4789-be65-df1485f1e172"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:45:10.311393 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.311371    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-dshm" (OuterVolumeSpecName: "dshm") pod "58be4fc6-222c-4789-be65-df1485f1e172" (UID: "58be4fc6-222c-4789-be65-df1485f1e172"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:45:10.311610 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.311389    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58be4fc6-222c-4789-be65-df1485f1e172-kube-api-access-zjjfg" (OuterVolumeSpecName: "kube-api-access-zjjfg") pod "58be4fc6-222c-4789-be65-df1485f1e172" (UID: "58be4fc6-222c-4789-be65-df1485f1e172"). InnerVolumeSpecName "kube-api-access-zjjfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:45:10.311672 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.311626    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58be4fc6-222c-4789-be65-df1485f1e172-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "58be4fc6-222c-4789-be65-df1485f1e172" (UID: "58be4fc6-222c-4789-be65-df1485f1e172"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:45:10.410703 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.410670    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjjfg\" (UniqueName: \"kubernetes.io/projected/58be4fc6-222c-4789-be65-df1485f1e172-kube-api-access-zjjfg\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:45:10.410703 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.410697    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58be4fc6-222c-4789-be65-df1485f1e172-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:45:10.410888 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.410714    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:45:10.410888 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:10.410725    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58be4fc6-222c-4789-be65-df1485f1e172-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:45:11.180249 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:11.180215    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"
Jul 22 15:45:11.180706 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:11.180220    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l" event={"ID":"58be4fc6-222c-4789-be65-df1485f1e172","Type":"ContainerDied","Data":"cf857a0ba8bf7cb7a48f6a4ec59fe3b60e9e56b843b423451fc8b89724d6817e"}
Jul 22 15:45:11.180706 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:11.180345    2560 scope.go:117] "RemoveContainer" containerID="66f8181b9468722b40537d6597f0f1084a2fbf4789cea030c4312e95e286a434"
Jul 22 15:45:11.200665 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:11.200642    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"]
Jul 22 15:45:11.204186 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:11.204166    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/auth-test-model-6f1c00-kserve-7854b6f999-rm82l"]
Jul 22 15:45:12.602150 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:12.602115    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58be4fc6-222c-4789-be65-df1485f1e172" path="/var/lib/kubelet/pods/58be4fc6-222c-4789-be65-df1485f1e172/volumes"
Jul 22 15:45:14.193707 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:14.193672    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:14.194215 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:14.193721    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:14.206516 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:14.206491    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:45:15.211891 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:45:15.211863    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:46:31.571991 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:31.571910    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"]
Jul 22 15:46:41.282868 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.282836    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"]
Jul 22 15:46:41.283319 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.283297    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58be4fc6-222c-4789-be65-df1485f1e172" containerName="main"
Jul 22 15:46:41.283319 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.283314    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="58be4fc6-222c-4789-be65-df1485f1e172" containerName="main"
Jul 22 15:46:41.283481 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.283403    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="58be4fc6-222c-4789-be65-df1485f1e172" containerName="main"
Jul 22 15:46:41.286441 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.286423    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.289382 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.289361    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6f1c00\"/\"rate-test-model-6f1c00-kserve-self-signed-certs\""
Jul 22 15:46:41.298278 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.298255    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"]
Jul 22 15:46:41.349534 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.349506    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-dshm\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.349670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.349552    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tls-certs\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.349670 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.349647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7shd\" (UniqueName: \"kubernetes.io/projected/ef278ba0-9a8d-49cc-b764-3cbdb6107295-kube-api-access-x7shd\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.349763 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.349684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tmp-dir\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.349763 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.349739    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-home\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.349854 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.349777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-model-cache\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.451026 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.450984    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-home\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.451217 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.451035    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-model-cache\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.451217 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.451064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-dshm\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.451217 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.451094    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tls-certs\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.451217 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.451138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7shd\" (UniqueName: \"kubernetes.io/projected/ef278ba0-9a8d-49cc-b764-3cbdb6107295-kube-api-access-x7shd\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.451217 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.451176    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tmp-dir\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.451518 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.451382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-home\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.451589 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.451521    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-model-cache\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.451589 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.451565    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tmp-dir\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.453477 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.453433    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-dshm\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.453579 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.453562    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tls-certs\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.460849 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.460830    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7shd\" (UniqueName: \"kubernetes.io/projected/ef278ba0-9a8d-49cc-b764-3cbdb6107295-kube-api-access-x7shd\") pod \"rate-test-model-6f1c00-kserve-589ff598b7-rtpsd\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") " pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.596062 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.595962    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:41.735276 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:41.735249    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"]
Jul 22 15:46:41.737261 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:46:41.737232    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef278ba0_9a8d_49cc_b764_3cbdb6107295.slice/crio-81acce1f065ad0f7fb36dbb1b85bd3aed579fcef4702ded9b072fcca8847f993 WatchSource:0}: Error finding container 81acce1f065ad0f7fb36dbb1b85bd3aed579fcef4702ded9b072fcca8847f993: Status 404 returned error can't find the container with id 81acce1f065ad0f7fb36dbb1b85bd3aed579fcef4702ded9b072fcca8847f993
Jul 22 15:46:42.656584 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:42.656550    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd" event={"ID":"ef278ba0-9a8d-49cc-b764-3cbdb6107295","Type":"ContainerStarted","Data":"3cd3cf1c14d524af6be647ee5eb8dfda22bac215c1563e06c899e3c6955d5bf8"}
Jul 22 15:46:42.656584 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:42.656592    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd" event={"ID":"ef278ba0-9a8d-49cc-b764-3cbdb6107295","Type":"ContainerStarted","Data":"81acce1f065ad0f7fb36dbb1b85bd3aed579fcef4702ded9b072fcca8847f993"}
Jul 22 15:46:42.674853 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:42.674801    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd" podStartSLOduration=1.6747841060000002 podStartE2EDuration="1.674784106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:46:42.673700637 +0000 UTC m=+2541.128414271" watchObservedRunningTime="2026-07-22 15:46:42.674784106 +0000 UTC m=+2541.129497721"
Jul 22 15:46:46.593388 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:46.593322    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb" podUID="94af53ff-12f0-4200-a415-3c962b7cc1fa" containerName="main" containerID="cri-o://eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe" gracePeriod=45
Jul 22 15:46:46.853871 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:46.853815    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:46:47.001356 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001324    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-model-cache\") pod \"94af53ff-12f0-4200-a415-3c962b7cc1fa\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") "
Jul 22 15:46:47.001549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001418    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-home\") pod \"94af53ff-12f0-4200-a415-3c962b7cc1fa\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") "
Jul 22 15:46:47.001549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001437    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94af53ff-12f0-4200-a415-3c962b7cc1fa-tls-certs\") pod \"94af53ff-12f0-4200-a415-3c962b7cc1fa\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") "
Jul 22 15:46:47.001549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001484    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-tmp-dir\") pod \"94af53ff-12f0-4200-a415-3c962b7cc1fa\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") "
Jul 22 15:46:47.001549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001510    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrqtg\" (UniqueName: \"kubernetes.io/projected/94af53ff-12f0-4200-a415-3c962b7cc1fa-kube-api-access-nrqtg\") pod \"94af53ff-12f0-4200-a415-3c962b7cc1fa\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") "
Jul 22 15:46:47.001549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001543    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-dshm\") pod \"94af53ff-12f0-4200-a415-3c962b7cc1fa\" (UID: \"94af53ff-12f0-4200-a415-3c962b7cc1fa\") "
Jul 22 15:46:47.001771 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001684    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "94af53ff-12f0-4200-a415-3c962b7cc1fa" (UID: "94af53ff-12f0-4200-a415-3c962b7cc1fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:46:47.001771 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001716    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-home" (OuterVolumeSpecName: "home") pod "94af53ff-12f0-4200-a415-3c962b7cc1fa" (UID: "94af53ff-12f0-4200-a415-3c962b7cc1fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:46:47.001771 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001753    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "94af53ff-12f0-4200-a415-3c962b7cc1fa" (UID: "94af53ff-12f0-4200-a415-3c962b7cc1fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:46:47.001935 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001865    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:46:47.001935 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001878    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:46:47.001935 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.001889    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:46:47.003630 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.003609    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94af53ff-12f0-4200-a415-3c962b7cc1fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "94af53ff-12f0-4200-a415-3c962b7cc1fa" (UID: "94af53ff-12f0-4200-a415-3c962b7cc1fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:46:47.003718 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.003698    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94af53ff-12f0-4200-a415-3c962b7cc1fa-kube-api-access-nrqtg" (OuterVolumeSpecName: "kube-api-access-nrqtg") pod "94af53ff-12f0-4200-a415-3c962b7cc1fa" (UID: "94af53ff-12f0-4200-a415-3c962b7cc1fa"). InnerVolumeSpecName "kube-api-access-nrqtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:46:47.003771 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.003721    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-dshm" (OuterVolumeSpecName: "dshm") pod "94af53ff-12f0-4200-a415-3c962b7cc1fa" (UID: "94af53ff-12f0-4200-a415-3c962b7cc1fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:46:47.102663 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.102616    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/94af53ff-12f0-4200-a415-3c962b7cc1fa-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:46:47.102663 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.102649    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrqtg\" (UniqueName: \"kubernetes.io/projected/94af53ff-12f0-4200-a415-3c962b7cc1fa-kube-api-access-nrqtg\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:46:47.102663 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.102659    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/94af53ff-12f0-4200-a415-3c962b7cc1fa-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:46:47.685478 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.685428    2560 generic.go:356] "Generic (PLEG): container finished" podID="94af53ff-12f0-4200-a415-3c962b7cc1fa" containerID="eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe" exitCode=0
Jul 22 15:46:47.685876 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.685499    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"
Jul 22 15:46:47.685876 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.685493    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb" event={"ID":"94af53ff-12f0-4200-a415-3c962b7cc1fa","Type":"ContainerDied","Data":"eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe"}
Jul 22 15:46:47.685876 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.685615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb" event={"ID":"94af53ff-12f0-4200-a415-3c962b7cc1fa","Type":"ContainerDied","Data":"00a11e44aaf902e5d304a0a6d09a4e0e8c3198db8b6e7f0e828259dbcddd23f1"}
Jul 22 15:46:47.685876 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.685638    2560 scope.go:117] "RemoveContainer" containerID="eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe"
Jul 22 15:46:47.694623 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.694604    2560 scope.go:117] "RemoveContainer" containerID="eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe"
Jul 22 15:46:47.694949 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:46:47.694925    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe\": container with ID starting with eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe not found: ID does not exist" containerID="eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe"
Jul 22 15:46:47.695037 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.694959    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe"} err="failed to get container status \"eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe\": rpc error: code = NotFound desc = could not find container \"eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe\": container with ID starting with eea1aea3ce5a06b19f73a13a5c8cb30f8e9284d1310c3ec27648e392501029fe not found: ID does not exist"
Jul 22 15:46:47.709930 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.709901    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"]
Jul 22 15:46:47.716426 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:47.716399    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/sub-test-model-6f1c00-kserve-6bb9f8cb99-5hhpb"]
Jul 22 15:46:48.601937 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:48.601901    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94af53ff-12f0-4200-a415-3c962b7cc1fa" path="/var/lib/kubelet/pods/94af53ff-12f0-4200-a415-3c962b7cc1fa/volumes"
Jul 22 15:46:51.596528 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:51.596485    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:51.596528 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:51.596539    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:51.609013 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:51.608987    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:46:51.717159 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:46:51.717133    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:47:22.412028 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:22.412000    2560 scope.go:117] "RemoveContainer" containerID="8ea11791ac0726087d8f4e85be61b08cca9be91415910dfcbbfde57c1a70f19d"
Jul 22 15:47:22.420433 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:22.420415    2560 scope.go:117] "RemoveContainer" containerID="cd6318359b5dcba17a1a084cbfa32c82783ff56799c6e063a4317edeb83bcac8"
Jul 22 15:47:51.858354 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:51.858277    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"]
Jul 22 15:47:57.501203 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.501168    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7643-140394085079728"]
Jul 22 15:47:57.501692 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.501596    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94af53ff-12f0-4200-a415-3c962b7cc1fa" containerName="main"
Jul 22 15:47:57.501692 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.501609    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="94af53ff-12f0-4200-a415-3c962b7cc1fa" containerName="main"
Jul 22 15:47:57.501692 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.501679    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="94af53ff-12f0-4200-a415-3c962b7cc1fa" containerName="main"
Jul 22 15:47:57.504997 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.504981    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085079728"
Jul 22 15:47:57.507567 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.507547    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-r9dl9\""
Jul 22 15:47:57.511906 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.511886    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085079728"]
Jul 22 15:47:57.562944 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.562902    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5p87\" (UniqueName: \"kubernetes.io/projected/fe06efbc-4ab8-4341-9d69-b1db27e0736d-kube-api-access-j5p87\") pod \"test-curl-7643-140394085079728\" (UID: \"fe06efbc-4ab8-4341-9d69-b1db27e0736d\") " pod="opendatahub/test-curl-7643-140394085079728"
Jul 22 15:47:57.663535 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.663436    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5p87\" (UniqueName: \"kubernetes.io/projected/fe06efbc-4ab8-4341-9d69-b1db27e0736d-kube-api-access-j5p87\") pod \"test-curl-7643-140394085079728\" (UID: \"fe06efbc-4ab8-4341-9d69-b1db27e0736d\") " pod="opendatahub/test-curl-7643-140394085079728"
Jul 22 15:47:57.671843 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.671810    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5p87\" (UniqueName: \"kubernetes.io/projected/fe06efbc-4ab8-4341-9d69-b1db27e0736d-kube-api-access-j5p87\") pod \"test-curl-7643-140394085079728\" (UID: \"fe06efbc-4ab8-4341-9d69-b1db27e0736d\") " pod="opendatahub/test-curl-7643-140394085079728"
Jul 22 15:47:57.815330 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.815234    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085079728"
Jul 22 15:47:57.948629 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.948606    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085079728"]
Jul 22 15:47:57.950793 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:47:57.950764    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe06efbc_4ab8_4341_9d69_b1db27e0736d.slice/crio-f05f2ce44e1afe243028165777466b65b3e2a0238b37c3039c726473528d401d WatchSource:0}: Error finding container f05f2ce44e1afe243028165777466b65b3e2a0238b37c3039c726473528d401d: Status 404 returned error can't find the container with id f05f2ce44e1afe243028165777466b65b3e2a0238b37c3039c726473528d401d
Jul 22 15:47:57.952209 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:57.952188    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 15:47:58.033522 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:58.033483    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085079728" event={"ID":"fe06efbc-4ab8-4341-9d69-b1db27e0736d","Type":"ContainerStarted","Data":"f05f2ce44e1afe243028165777466b65b3e2a0238b37c3039c726473528d401d"}
Jul 22 15:47:59.039484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:59.039373    2560 generic.go:356] "Generic (PLEG): container finished" podID="fe06efbc-4ab8-4341-9d69-b1db27e0736d" containerID="9439c5aac622d234461b4c419d84cb6d3115278aea0095e862af57ab46b33e20" exitCode=0
Jul 22 15:47:59.039871 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:47:59.039500    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085079728" event={"ID":"fe06efbc-4ab8-4341-9d69-b1db27e0736d","Type":"ContainerDied","Data":"9439c5aac622d234461b4c419d84cb6d3115278aea0095e862af57ab46b33e20"}
Jul 22 15:48:00.179357 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.179329    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085079728"
Jul 22 15:48:00.220901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.220870    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7643-140394085079728_fe06efbc-4ab8-4341-9d69-b1db27e0736d/test-curl-7643-140394085079728/0.log"
Jul 22 15:48:00.244194 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.244159    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7643-140394085079728"]
Jul 22 15:48:00.248825 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.248797    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7643-140394085079728"]
Jul 22 15:48:00.289178 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.289144    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5p87\" (UniqueName: \"kubernetes.io/projected/fe06efbc-4ab8-4341-9d69-b1db27e0736d-kube-api-access-j5p87\") pod \"fe06efbc-4ab8-4341-9d69-b1db27e0736d\" (UID: \"fe06efbc-4ab8-4341-9d69-b1db27e0736d\") "
Jul 22 15:48:00.291306 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.291274    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe06efbc-4ab8-4341-9d69-b1db27e0736d-kube-api-access-j5p87" (OuterVolumeSpecName: "kube-api-access-j5p87") pod "fe06efbc-4ab8-4341-9d69-b1db27e0736d" (UID: "fe06efbc-4ab8-4341-9d69-b1db27e0736d"). InnerVolumeSpecName "kube-api-access-j5p87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:48:00.390136 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.390100    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5p87\" (UniqueName: \"kubernetes.io/projected/fe06efbc-4ab8-4341-9d69-b1db27e0736d-kube-api-access-j5p87\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:00.423049 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.423007    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:00.423437 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.423422    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe06efbc-4ab8-4341-9d69-b1db27e0736d" containerName="test-curl-7643-140394085079728"
Jul 22 15:48:00.423519 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.423439    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe06efbc-4ab8-4341-9d69-b1db27e0736d" containerName="test-curl-7643-140394085079728"
Jul 22 15:48:00.423557 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.423521    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe06efbc-4ab8-4341-9d69-b1db27e0736d" containerName="test-curl-7643-140394085079728"
Jul 22 15:48:00.426576 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.426556    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:00.431294 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.431267    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:00.490828 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.490785    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpq2s\" (UniqueName: \"kubernetes.io/projected/804e2a3f-66c1-445f-bffb-d5f172d08c58-kube-api-access-xpq2s\") pod \"test-curl-7643-140394085008272\" (UID: \"804e2a3f-66c1-445f-bffb-d5f172d08c58\") " pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:00.592258 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.592146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpq2s\" (UniqueName: \"kubernetes.io/projected/804e2a3f-66c1-445f-bffb-d5f172d08c58-kube-api-access-xpq2s\") pod \"test-curl-7643-140394085008272\" (UID: \"804e2a3f-66c1-445f-bffb-d5f172d08c58\") " pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:00.601515 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.601482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpq2s\" (UniqueName: \"kubernetes.io/projected/804e2a3f-66c1-445f-bffb-d5f172d08c58-kube-api-access-xpq2s\") pod \"test-curl-7643-140394085008272\" (UID: \"804e2a3f-66c1-445f-bffb-d5f172d08c58\") " pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:00.602742 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.602713    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe06efbc-4ab8-4341-9d69-b1db27e0736d" path="/var/lib/kubelet/pods/fe06efbc-4ab8-4341-9d69-b1db27e0736d/volumes"
Jul 22 15:48:00.737988 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.737946    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:00.879081 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:00.878994    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:00.882300 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:48:00.882271    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod804e2a3f_66c1_445f_bffb_d5f172d08c58.slice/crio-ab64b866cf28323270337b7dfceca6665f5f4a9ef5c21d4fefe4f9c5fef59635 WatchSource:0}: Error finding container ab64b866cf28323270337b7dfceca6665f5f4a9ef5c21d4fefe4f9c5fef59635: Status 404 returned error can't find the container with id ab64b866cf28323270337b7dfceca6665f5f4a9ef5c21d4fefe4f9c5fef59635
Jul 22 15:48:01.050579 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:01.050539    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085008272" event={"ID":"804e2a3f-66c1-445f-bffb-d5f172d08c58","Type":"ContainerStarted","Data":"ab64b866cf28323270337b7dfceca6665f5f4a9ef5c21d4fefe4f9c5fef59635"}
Jul 22 15:48:01.051760 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:01.051740    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085079728"
Jul 22 15:48:01.051901 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:01.051782    2560 scope.go:117] "RemoveContainer" containerID="9439c5aac622d234461b4c419d84cb6d3115278aea0095e862af57ab46b33e20"
Jul 22 15:48:02.058577 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:02.058545    2560 generic.go:356] "Generic (PLEG): container finished" podID="804e2a3f-66c1-445f-bffb-d5f172d08c58" containerID="d73c39903946691c372a4b633a3e9fc44b5bb1b17a65c862e089c67da1395978" exitCode=0
Jul 22 15:48:02.059007 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:02.058633    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085008272" event={"ID":"804e2a3f-66c1-445f-bffb-d5f172d08c58","Type":"ContainerDied","Data":"d73c39903946691c372a4b633a3e9fc44b5bb1b17a65c862e089c67da1395978"}
Jul 22 15:48:03.199607 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.199571    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:03.220233 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.220203    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7643-140394085008272_804e2a3f-66c1-445f-bffb-d5f172d08c58/test-curl-7643-140394085008272/0.log"
Jul 22 15:48:03.243357 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.243321    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:03.247324 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.247302    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:03.320420 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.320338    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpq2s\" (UniqueName: \"kubernetes.io/projected/804e2a3f-66c1-445f-bffb-d5f172d08c58-kube-api-access-xpq2s\") pod \"804e2a3f-66c1-445f-bffb-d5f172d08c58\" (UID: \"804e2a3f-66c1-445f-bffb-d5f172d08c58\") "
Jul 22 15:48:03.322433 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.322403    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/804e2a3f-66c1-445f-bffb-d5f172d08c58-kube-api-access-xpq2s" (OuterVolumeSpecName: "kube-api-access-xpq2s") pod "804e2a3f-66c1-445f-bffb-d5f172d08c58" (UID: "804e2a3f-66c1-445f-bffb-d5f172d08c58"). InnerVolumeSpecName "kube-api-access-xpq2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:48:03.386792 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.386753    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:03.387252 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.387231    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="804e2a3f-66c1-445f-bffb-d5f172d08c58" containerName="test-curl-7643-140394085008272"
Jul 22 15:48:03.387351 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.387253    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="804e2a3f-66c1-445f-bffb-d5f172d08c58" containerName="test-curl-7643-140394085008272"
Jul 22 15:48:03.387409 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.387349    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="804e2a3f-66c1-445f-bffb-d5f172d08c58" containerName="test-curl-7643-140394085008272"
Jul 22 15:48:03.390542 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.390522    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:03.396097 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.396075    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:03.421950 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.421914    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpq2s\" (UniqueName: \"kubernetes.io/projected/804e2a3f-66c1-445f-bffb-d5f172d08c58-kube-api-access-xpq2s\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:03.523398 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.523356    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b965s\" (UniqueName: \"kubernetes.io/projected/74cdbcb3-badc-4b5a-90c6-24b775954a21-kube-api-access-b965s\") pod \"test-curl-7643-140394085008272\" (UID: \"74cdbcb3-badc-4b5a-90c6-24b775954a21\") " pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:03.624908 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.624813    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b965s\" (UniqueName: \"kubernetes.io/projected/74cdbcb3-badc-4b5a-90c6-24b775954a21-kube-api-access-b965s\") pod \"test-curl-7643-140394085008272\" (UID: \"74cdbcb3-badc-4b5a-90c6-24b775954a21\") " pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:03.634735 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.634705    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b965s\" (UniqueName: \"kubernetes.io/projected/74cdbcb3-badc-4b5a-90c6-24b775954a21-kube-api-access-b965s\") pod \"test-curl-7643-140394085008272\" (UID: \"74cdbcb3-badc-4b5a-90c6-24b775954a21\") " pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:03.701832 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.701784    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:03.840690 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:03.840661    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:03.842792 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:48:03.842765    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74cdbcb3_badc_4b5a_90c6_24b775954a21.slice/crio-3a6493f20fe96b289f9bab452139e98a4887f4e0d5ee148c6a90313be70409b3 WatchSource:0}: Error finding container 3a6493f20fe96b289f9bab452139e98a4887f4e0d5ee148c6a90313be70409b3: Status 404 returned error can't find the container with id 3a6493f20fe96b289f9bab452139e98a4887f4e0d5ee148c6a90313be70409b3
Jul 22 15:48:04.072300 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:04.072260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085008272" event={"ID":"74cdbcb3-badc-4b5a-90c6-24b775954a21","Type":"ContainerStarted","Data":"3a6493f20fe96b289f9bab452139e98a4887f4e0d5ee148c6a90313be70409b3"}
Jul 22 15:48:04.073379 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:04.073356    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:04.073532 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:04.073399    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab64b866cf28323270337b7dfceca6665f5f4a9ef5c21d4fefe4f9c5fef59635"
Jul 22 15:48:04.085311 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:04.085279    2560 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7643-140394085008272" oldPodUID="804e2a3f-66c1-445f-bffb-d5f172d08c58" podUID="74cdbcb3-badc-4b5a-90c6-24b775954a21"
Jul 22 15:48:04.601872 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:04.601839    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="804e2a3f-66c1-445f-bffb-d5f172d08c58" path="/var/lib/kubelet/pods/804e2a3f-66c1-445f-bffb-d5f172d08c58/volumes"
Jul 22 15:48:05.080585 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:05.080550    2560 generic.go:356] "Generic (PLEG): container finished" podID="74cdbcb3-badc-4b5a-90c6-24b775954a21" containerID="10dabc8353fc122ff066b292d04d840c0e901de729a5f8b9c63c8c5ab70d3dd1" exitCode=0
Jul 22 15:48:05.080757 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:05.080629    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085008272" event={"ID":"74cdbcb3-badc-4b5a-90c6-24b775954a21","Type":"ContainerDied","Data":"10dabc8353fc122ff066b292d04d840c0e901de729a5f8b9c63c8c5ab70d3dd1"}
Jul 22 15:48:06.228123 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.228098    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:06.248400 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.248364    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7643-140394085008272_74cdbcb3-badc-4b5a-90c6-24b775954a21/test-curl-7643-140394085008272/0.log"
Jul 22 15:48:06.278101 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.278071    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:06.281462 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.281426    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7643-140394085008272"]
Jul 22 15:48:06.350707 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.350629    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b965s\" (UniqueName: \"kubernetes.io/projected/74cdbcb3-badc-4b5a-90c6-24b775954a21-kube-api-access-b965s\") pod \"74cdbcb3-badc-4b5a-90c6-24b775954a21\" (UID: \"74cdbcb3-badc-4b5a-90c6-24b775954a21\") "
Jul 22 15:48:06.352649 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.352620    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74cdbcb3-badc-4b5a-90c6-24b775954a21-kube-api-access-b965s" (OuterVolumeSpecName: "kube-api-access-b965s") pod "74cdbcb3-badc-4b5a-90c6-24b775954a21" (UID: "74cdbcb3-badc-4b5a-90c6-24b775954a21"). InnerVolumeSpecName "kube-api-access-b965s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:48:06.412965 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.412937    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:06.413329 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.413317    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74cdbcb3-badc-4b5a-90c6-24b775954a21" containerName="test-curl-7643-140394085008272"
Jul 22 15:48:06.413372 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.413331    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="74cdbcb3-badc-4b5a-90c6-24b775954a21" containerName="test-curl-7643-140394085008272"
Jul 22 15:48:06.413428 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.413419    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="74cdbcb3-badc-4b5a-90c6-24b775954a21" containerName="test-curl-7643-140394085008272"
Jul 22 15:48:06.416346 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.416330    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:06.424002 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.423976    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:06.452552 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.452515    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b965s\" (UniqueName: \"kubernetes.io/projected/74cdbcb3-badc-4b5a-90c6-24b775954a21-kube-api-access-b965s\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:06.554562 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.554514    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2hg9\" (UniqueName: \"kubernetes.io/projected/0b7dec04-ac8e-4c93-94e1-4f5359c4123e-kube-api-access-n2hg9\") pod \"test-curl-7643-140394085009280\" (UID: \"0b7dec04-ac8e-4c93-94e1-4f5359c4123e\") " pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:06.602283 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.602182    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74cdbcb3-badc-4b5a-90c6-24b775954a21" path="/var/lib/kubelet/pods/74cdbcb3-badc-4b5a-90c6-24b775954a21/volumes"
Jul 22 15:48:06.655671 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.655639    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2hg9\" (UniqueName: \"kubernetes.io/projected/0b7dec04-ac8e-4c93-94e1-4f5359c4123e-kube-api-access-n2hg9\") pod \"test-curl-7643-140394085009280\" (UID: \"0b7dec04-ac8e-4c93-94e1-4f5359c4123e\") " pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:06.666612 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.666580    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2hg9\" (UniqueName: \"kubernetes.io/projected/0b7dec04-ac8e-4c93-94e1-4f5359c4123e-kube-api-access-n2hg9\") pod \"test-curl-7643-140394085009280\" (UID: \"0b7dec04-ac8e-4c93-94e1-4f5359c4123e\") " pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:06.727267 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.727230    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:06.865518 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.865491    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:06.868276 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:48:06.868247    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b7dec04_ac8e_4c93_94e1_4f5359c4123e.slice/crio-3b612264615d9c34bd631529d02c8890662e620d62b3952f9425e6e61d4d2113 WatchSource:0}: Error finding container 3b612264615d9c34bd631529d02c8890662e620d62b3952f9425e6e61d4d2113: Status 404 returned error can't find the container with id 3b612264615d9c34bd631529d02c8890662e620d62b3952f9425e6e61d4d2113
Jul 22 15:48:06.878415 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:06.878360    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd" podUID="ef278ba0-9a8d-49cc-b764-3cbdb6107295" containerName="main" containerID="cri-o://3cd3cf1c14d524af6be647ee5eb8dfda22bac215c1563e06c899e3c6955d5bf8" gracePeriod=45
Jul 22 15:48:07.092549 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.092511    2560 generic.go:356] "Generic (PLEG): container finished" podID="ef278ba0-9a8d-49cc-b764-3cbdb6107295" containerID="3cd3cf1c14d524af6be647ee5eb8dfda22bac215c1563e06c899e3c6955d5bf8" exitCode=0
Jul 22 15:48:07.092702 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.092604    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd" event={"ID":"ef278ba0-9a8d-49cc-b764-3cbdb6107295","Type":"ContainerDied","Data":"3cd3cf1c14d524af6be647ee5eb8dfda22bac215c1563e06c899e3c6955d5bf8"}
Jul 22 15:48:07.093832 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.093808    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085009280" event={"ID":"0b7dec04-ac8e-4c93-94e1-4f5359c4123e","Type":"ContainerStarted","Data":"3b612264615d9c34bd631529d02c8890662e620d62b3952f9425e6e61d4d2113"}
Jul 22 15:48:07.094998 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.094974    2560 scope.go:117] "RemoveContainer" containerID="10dabc8353fc122ff066b292d04d840c0e901de729a5f8b9c63c8c5ab70d3dd1"
Jul 22 15:48:07.094998 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.094993    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085008272"
Jul 22 15:48:07.120511 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.120439    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:48:07.262262 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262234    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tmp-dir\") pod \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") "
Jul 22 15:48:07.262262 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262266    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-model-cache\") pod \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") "
Jul 22 15:48:07.262747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262344    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7shd\" (UniqueName: \"kubernetes.io/projected/ef278ba0-9a8d-49cc-b764-3cbdb6107295-kube-api-access-x7shd\") pod \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") "
Jul 22 15:48:07.262747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262367    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tls-certs\") pod \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") "
Jul 22 15:48:07.262747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262391    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-dshm\") pod \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") "
Jul 22 15:48:07.262747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262499    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-home\") pod \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\" (UID: \"ef278ba0-9a8d-49cc-b764-3cbdb6107295\") "
Jul 22 15:48:07.262747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262508    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef278ba0-9a8d-49cc-b764-3cbdb6107295" (UID: "ef278ba0-9a8d-49cc-b764-3cbdb6107295"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:48:07.262747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262634    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef278ba0-9a8d-49cc-b764-3cbdb6107295" (UID: "ef278ba0-9a8d-49cc-b764-3cbdb6107295"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:48:07.262991 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262785    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-home" (OuterVolumeSpecName: "home") pod "ef278ba0-9a8d-49cc-b764-3cbdb6107295" (UID: "ef278ba0-9a8d-49cc-b764-3cbdb6107295"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:48:07.262991 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262872    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tmp-dir\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:07.262991 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262894    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-model-cache\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:07.262991 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.262908    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-home\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:07.264443 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.264417    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef278ba0-9a8d-49cc-b764-3cbdb6107295" (UID: "ef278ba0-9a8d-49cc-b764-3cbdb6107295"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:48:07.264443 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.264440    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-dshm" (OuterVolumeSpecName: "dshm") pod "ef278ba0-9a8d-49cc-b764-3cbdb6107295" (UID: "ef278ba0-9a8d-49cc-b764-3cbdb6107295"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 15:48:07.264606 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.264480    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef278ba0-9a8d-49cc-b764-3cbdb6107295-kube-api-access-x7shd" (OuterVolumeSpecName: "kube-api-access-x7shd") pod "ef278ba0-9a8d-49cc-b764-3cbdb6107295" (UID: "ef278ba0-9a8d-49cc-b764-3cbdb6107295"). InnerVolumeSpecName "kube-api-access-x7shd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:48:07.363427 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.363388    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7shd\" (UniqueName: \"kubernetes.io/projected/ef278ba0-9a8d-49cc-b764-3cbdb6107295-kube-api-access-x7shd\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:07.363427 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.363424    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef278ba0-9a8d-49cc-b764-3cbdb6107295-tls-certs\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:07.363427 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:07.363436    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef278ba0-9a8d-49cc-b764-3cbdb6107295-dshm\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:08.101220 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:08.101177    2560 generic.go:356] "Generic (PLEG): container finished" podID="0b7dec04-ac8e-4c93-94e1-4f5359c4123e" containerID="789b5734d107821333d8d3993f71085636da1efa6da69faefe64c025cad0cbaf" exitCode=0
Jul 22 15:48:08.101424 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:08.101219    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085009280" event={"ID":"0b7dec04-ac8e-4c93-94e1-4f5359c4123e","Type":"ContainerDied","Data":"789b5734d107821333d8d3993f71085636da1efa6da69faefe64c025cad0cbaf"}
Jul 22 15:48:08.103553 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:08.103531    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"
Jul 22 15:48:08.103553 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:08.103539    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd" event={"ID":"ef278ba0-9a8d-49cc-b764-3cbdb6107295","Type":"ContainerDied","Data":"81acce1f065ad0f7fb36dbb1b85bd3aed579fcef4702ded9b072fcca8847f993"}
Jul 22 15:48:08.103728 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:08.103585    2560 scope.go:117] "RemoveContainer" containerID="3cd3cf1c14d524af6be647ee5eb8dfda22bac215c1563e06c899e3c6955d5bf8"
Jul 22 15:48:08.136771 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:08.136723    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"]
Jul 22 15:48:08.142619 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:08.142587    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6f1c00/rate-test-model-6f1c00-kserve-589ff598b7-rtpsd"]
Jul 22 15:48:08.602373 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:08.602341    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef278ba0-9a8d-49cc-b764-3cbdb6107295" path="/var/lib/kubelet/pods/ef278ba0-9a8d-49cc-b764-3cbdb6107295/volumes"
Jul 22 15:48:09.245858 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.245830    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:09.266838 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.266811    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7643-140394085009280_0b7dec04-ac8e-4c93-94e1-4f5359c4123e/test-curl-7643-140394085009280/0.log"
Jul 22 15:48:09.291046 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.291008    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:09.293547 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.293520    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:09.383731 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.383608    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2hg9\" (UniqueName: \"kubernetes.io/projected/0b7dec04-ac8e-4c93-94e1-4f5359c4123e-kube-api-access-n2hg9\") pod \"0b7dec04-ac8e-4c93-94e1-4f5359c4123e\" (UID: \"0b7dec04-ac8e-4c93-94e1-4f5359c4123e\") "
Jul 22 15:48:09.385711 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.385670    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7dec04-ac8e-4c93-94e1-4f5359c4123e-kube-api-access-n2hg9" (OuterVolumeSpecName: "kube-api-access-n2hg9") pod "0b7dec04-ac8e-4c93-94e1-4f5359c4123e" (UID: "0b7dec04-ac8e-4c93-94e1-4f5359c4123e"). InnerVolumeSpecName "kube-api-access-n2hg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:48:09.416575 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.416538    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:09.416970 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.416954    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef278ba0-9a8d-49cc-b764-3cbdb6107295" containerName="main"
Jul 22 15:48:09.417015 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.416972    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef278ba0-9a8d-49cc-b764-3cbdb6107295" containerName="main"
Jul 22 15:48:09.417015 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.416996    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b7dec04-ac8e-4c93-94e1-4f5359c4123e" containerName="test-curl-7643-140394085009280"
Jul 22 15:48:09.417015 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.417002    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b7dec04-ac8e-4c93-94e1-4f5359c4123e" containerName="test-curl-7643-140394085009280"
Jul 22 15:48:09.417113 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.417066    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef278ba0-9a8d-49cc-b764-3cbdb6107295" containerName="main"
Jul 22 15:48:09.417113 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.417078    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b7dec04-ac8e-4c93-94e1-4f5359c4123e" containerName="test-curl-7643-140394085009280"
Jul 22 15:48:09.420334 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.420317    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:09.427587 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.427559    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:09.484991 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.484955    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bsk2\" (UniqueName: \"kubernetes.io/projected/5df87615-ed9c-4dca-b921-557f2bb9843c-kube-api-access-7bsk2\") pod \"test-curl-7643-140394085009280\" (UID: \"5df87615-ed9c-4dca-b921-557f2bb9843c\") " pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:09.485169 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.485044    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2hg9\" (UniqueName: \"kubernetes.io/projected/0b7dec04-ac8e-4c93-94e1-4f5359c4123e-kube-api-access-n2hg9\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:09.585699 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.585657    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bsk2\" (UniqueName: \"kubernetes.io/projected/5df87615-ed9c-4dca-b921-557f2bb9843c-kube-api-access-7bsk2\") pod \"test-curl-7643-140394085009280\" (UID: \"5df87615-ed9c-4dca-b921-557f2bb9843c\") " pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:09.594201 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.594172    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bsk2\" (UniqueName: \"kubernetes.io/projected/5df87615-ed9c-4dca-b921-557f2bb9843c-kube-api-access-7bsk2\") pod \"test-curl-7643-140394085009280\" (UID: \"5df87615-ed9c-4dca-b921-557f2bb9843c\") " pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:09.731394 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.731352    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:09.869127 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:09.869101    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:09.870336 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:48:09.870305    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5df87615_ed9c_4dca_b921_557f2bb9843c.slice/crio-f1f45d2862ed7901ee02a248a256dabe3dccb5fea6d6f5fbd5b564109b5d4e54 WatchSource:0}: Error finding container f1f45d2862ed7901ee02a248a256dabe3dccb5fea6d6f5fbd5b564109b5d4e54: Status 404 returned error can't find the container with id f1f45d2862ed7901ee02a248a256dabe3dccb5fea6d6f5fbd5b564109b5d4e54
Jul 22 15:48:10.115696 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:10.115597    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085009280" event={"ID":"5df87615-ed9c-4dca-b921-557f2bb9843c","Type":"ContainerStarted","Data":"f1f45d2862ed7901ee02a248a256dabe3dccb5fea6d6f5fbd5b564109b5d4e54"}
Jul 22 15:48:10.116835 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:10.116813    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:10.116835 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:10.116826    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b612264615d9c34bd631529d02c8890662e620d62b3952f9425e6e61d4d2113"
Jul 22 15:48:10.134768 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:10.134740    2560 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7643-140394085009280" oldPodUID="0b7dec04-ac8e-4c93-94e1-4f5359c4123e" podUID="5df87615-ed9c-4dca-b921-557f2bb9843c"
Jul 22 15:48:10.602226 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:10.602194    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b7dec04-ac8e-4c93-94e1-4f5359c4123e" path="/var/lib/kubelet/pods/0b7dec04-ac8e-4c93-94e1-4f5359c4123e/volumes"
Jul 22 15:48:11.122879 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:11.122845    2560 generic.go:356] "Generic (PLEG): container finished" podID="5df87615-ed9c-4dca-b921-557f2bb9843c" containerID="70214dd40a3781171d4f8642f1d593f1ab07d00a53b8f068feeb8f58872595c6" exitCode=0
Jul 22 15:48:11.123269 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:11.122934    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7643-140394085009280" event={"ID":"5df87615-ed9c-4dca-b921-557f2bb9843c","Type":"ContainerDied","Data":"70214dd40a3781171d4f8642f1d593f1ab07d00a53b8f068feeb8f58872595c6"}
Jul 22 15:48:12.272023 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.271994    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:12.293702 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.293656    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7643-140394085009280_5df87615-ed9c-4dca-b921-557f2bb9843c/test-curl-7643-140394085009280/0.log"
Jul 22 15:48:12.315528 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.315498    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:12.320818 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.320789    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7643-140394085009280"]
Jul 22 15:48:12.414994 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.414962    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bsk2\" (UniqueName: \"kubernetes.io/projected/5df87615-ed9c-4dca-b921-557f2bb9843c-kube-api-access-7bsk2\") pod \"5df87615-ed9c-4dca-b921-557f2bb9843c\" (UID: \"5df87615-ed9c-4dca-b921-557f2bb9843c\") "
Jul 22 15:48:12.417167 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.417129    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df87615-ed9c-4dca-b921-557f2bb9843c-kube-api-access-7bsk2" (OuterVolumeSpecName: "kube-api-access-7bsk2") pod "5df87615-ed9c-4dca-b921-557f2bb9843c" (UID: "5df87615-ed9c-4dca-b921-557f2bb9843c"). InnerVolumeSpecName "kube-api-access-7bsk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:48:12.511604 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.511564    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"]
Jul 22 15:48:12.511973 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.511957    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df87615-ed9c-4dca-b921-557f2bb9843c" containerName="test-curl-7643-140394085009280"
Jul 22 15:48:12.511973 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.511973    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df87615-ed9c-4dca-b921-557f2bb9843c" containerName="test-curl-7643-140394085009280"
Jul 22 15:48:12.512153 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.512047    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df87615-ed9c-4dca-b921-557f2bb9843c" containerName="test-curl-7643-140394085009280"
Jul 22 15:48:12.515092 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.515068    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"
Jul 22 15:48:12.515753 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.515734    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bsk2\" (UniqueName: \"kubernetes.io/projected/5df87615-ed9c-4dca-b921-557f2bb9843c-kube-api-access-7bsk2\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:12.517942 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.517921    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:48:12.522925 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.522903    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"]
Jul 22 15:48:12.603080 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.603049    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5df87615-ed9c-4dca-b921-557f2bb9843c" path="/var/lib/kubelet/pods/5df87615-ed9c-4dca-b921-557f2bb9843c/volumes"
Jul 22 15:48:12.616529 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.616498    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf226a03-aaed-413f-90c2-576b8ff2b0a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7\" (UID: \"bf226a03-aaed-413f-90c2-576b8ff2b0a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"
Jul 22 15:48:12.717339 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.717241    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf226a03-aaed-413f-90c2-576b8ff2b0a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7\" (UID: \"bf226a03-aaed-413f-90c2-576b8ff2b0a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"
Jul 22 15:48:12.718213 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.718189    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf226a03-aaed-413f-90c2-576b8ff2b0a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7\" (UID: \"bf226a03-aaed-413f-90c2-576b8ff2b0a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"
Jul 22 15:48:12.827318 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.827282    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"
Jul 22 15:48:12.967462 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:12.967280    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"]
Jul 22 15:48:12.969779 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:48:12.969755    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf226a03_aaed_413f_90c2_576b8ff2b0a3.slice/crio-4261d7b9adb58a4a0e334f89dea11ede1f752974c872a193fafdd1bb2617d9ca WatchSource:0}: Error finding container 4261d7b9adb58a4a0e334f89dea11ede1f752974c872a193fafdd1bb2617d9ca: Status 404 returned error can't find the container with id 4261d7b9adb58a4a0e334f89dea11ede1f752974c872a193fafdd1bb2617d9ca
Jul 22 15:48:13.136056 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:13.136023    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7" event={"ID":"bf226a03-aaed-413f-90c2-576b8ff2b0a3","Type":"ContainerStarted","Data":"4261d7b9adb58a4a0e334f89dea11ede1f752974c872a193fafdd1bb2617d9ca"}
Jul 22 15:48:13.137297 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:13.137276    2560 scope.go:117] "RemoveContainer" containerID="70214dd40a3781171d4f8642f1d593f1ab07d00a53b8f068feeb8f58872595c6"
Jul 22 15:48:13.137408 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:13.137277    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7643-140394085009280"
Jul 22 15:48:14.145065 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:14.145031    2560 generic.go:356] "Generic (PLEG): container finished" podID="bf226a03-aaed-413f-90c2-576b8ff2b0a3" containerID="818726beaf62554c9d0c7eca1ea4b9b9267537e8a204f9b56e9126ff53d83b75" exitCode=0
Jul 22 15:48:14.145473 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:14.145094    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7" event={"ID":"bf226a03-aaed-413f-90c2-576b8ff2b0a3","Type":"ContainerDied","Data":"818726beaf62554c9d0c7eca1ea4b9b9267537e8a204f9b56e9126ff53d83b75"}
Jul 22 15:48:15.288090 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:15.288061    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"
Jul 22 15:48:15.441068 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:15.440982    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf226a03-aaed-413f-90c2-576b8ff2b0a3-maas-api-service-ca\") pod \"bf226a03-aaed-413f-90c2-576b8ff2b0a3\" (UID: \"bf226a03-aaed-413f-90c2-576b8ff2b0a3\") "
Jul 22 15:48:15.441350 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:15.441328    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf226a03-aaed-413f-90c2-576b8ff2b0a3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf226a03-aaed-413f-90c2-576b8ff2b0a3" (UID: "bf226a03-aaed-413f-90c2-576b8ff2b0a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:48:15.541738 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:15.541700    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf226a03-aaed-413f-90c2-576b8ff2b0a3-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:16.156728 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:16.156694    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7" event={"ID":"bf226a03-aaed-413f-90c2-576b8ff2b0a3","Type":"ContainerDied","Data":"4261d7b9adb58a4a0e334f89dea11ede1f752974c872a193fafdd1bb2617d9ca"}
Jul 22 15:48:16.156897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:16.156742    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4261d7b9adb58a4a0e334f89dea11ede1f752974c872a193fafdd1bb2617d9ca"
Jul 22 15:48:16.156897 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:16.156701    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"
Jul 22 15:48:22.449869 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:22.449840    2560 scope.go:117] "RemoveContainer" containerID="6fe7a4a3e8e8316aed9eea0d174cc506be40b05324256fe71887860df651ee46"
Jul 22 15:48:23.171469 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.159008    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"]
Jul 22 15:48:23.171469 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.159239    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t" podUID="d2cd472f-17a8-4642-8760-4df286ef822b" containerName="maas-api" containerID="cri-o://46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118" gracePeriod=30
Jul 22 15:48:23.203272 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.203244    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v"]
Jul 22 15:48:23.207073 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.207046    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-6f1c00-29745585-l577v"]
Jul 22 15:48:23.420783 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.420760    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:48:23.521579 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.521500    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t8s8\" (UniqueName: \"kubernetes.io/projected/d2cd472f-17a8-4642-8760-4df286ef822b-kube-api-access-6t8s8\") pod \"d2cd472f-17a8-4642-8760-4df286ef822b\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") "
Jul 22 15:48:23.521579 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.521540    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd472f-17a8-4642-8760-4df286ef822b-maas-api-tls\") pod \"d2cd472f-17a8-4642-8760-4df286ef822b\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") "
Jul 22 15:48:23.522061 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.521646    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd472f-17a8-4642-8760-4df286ef822b-openshift-service-ca\") pod \"d2cd472f-17a8-4642-8760-4df286ef822b\" (UID: \"d2cd472f-17a8-4642-8760-4df286ef822b\") "
Jul 22 15:48:23.522061 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.522034    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2cd472f-17a8-4642-8760-4df286ef822b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d2cd472f-17a8-4642-8760-4df286ef822b" (UID: "d2cd472f-17a8-4642-8760-4df286ef822b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:48:23.523611 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.523587    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2cd472f-17a8-4642-8760-4df286ef822b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d2cd472f-17a8-4642-8760-4df286ef822b" (UID: "d2cd472f-17a8-4642-8760-4df286ef822b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:48:23.523611 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.523602    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2cd472f-17a8-4642-8760-4df286ef822b-kube-api-access-6t8s8" (OuterVolumeSpecName: "kube-api-access-6t8s8") pod "d2cd472f-17a8-4642-8760-4df286ef822b" (UID: "d2cd472f-17a8-4642-8760-4df286ef822b"). InnerVolumeSpecName "kube-api-access-6t8s8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:48:23.623212 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.623180    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd472f-17a8-4642-8760-4df286ef822b-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:23.623212 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.623207    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t8s8\" (UniqueName: \"kubernetes.io/projected/d2cd472f-17a8-4642-8760-4df286ef822b-kube-api-access-6t8s8\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:23.623412 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:23.623218    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd472f-17a8-4642-8760-4df286ef822b-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:24.198776 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.198738    2560 generic.go:356] "Generic (PLEG): container finished" podID="d2cd472f-17a8-4642-8760-4df286ef822b" containerID="46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118" exitCode=0
Jul 22 15:48:24.198962 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.198797    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"
Jul 22 15:48:24.198962 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.198798    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t" event={"ID":"d2cd472f-17a8-4642-8760-4df286ef822b","Type":"ContainerDied","Data":"46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118"}
Jul 22 15:48:24.198962 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.198854    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t" event={"ID":"d2cd472f-17a8-4642-8760-4df286ef822b","Type":"ContainerDied","Data":"6c755a5a3a81858007adde9ef045d00a2228b05c7dfd5e987ea9732383bf1677"}
Jul 22 15:48:24.198962 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.198884    2560 scope.go:117] "RemoveContainer" containerID="46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118"
Jul 22 15:48:24.207857 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.207837    2560 scope.go:117] "RemoveContainer" containerID="46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118"
Jul 22 15:48:24.208124 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:48:24.208102    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118\": container with ID starting with 46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118 not found: ID does not exist" containerID="46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118"
Jul 22 15:48:24.208192 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.208139    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118"} err="failed to get container status \"46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118\": rpc error: code = NotFound desc = could not find container \"46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118\": container with ID starting with 46d2d38438cd748d984e6579a6d21ceb849ab488ed10be7d21da22e52e822118 not found: ID does not exist"
Jul 22 15:48:24.225004 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.224974    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"]
Jul 22 15:48:24.230425 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.230402    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-6f1c00-55b8ddd8b8-m9d9t"]
Jul 22 15:48:24.602384 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.602280    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c4de1a6-cb19-42cd-a64b-3e76b3274d2d" path="/var/lib/kubelet/pods/5c4de1a6-cb19-42cd-a64b-3e76b3274d2d/volumes"
Jul 22 15:48:24.602908 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:24.602853    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2cd472f-17a8-4642-8760-4df286ef822b" path="/var/lib/kubelet/pods/d2cd472f-17a8-4642-8760-4df286ef822b/volumes"
Jul 22 15:48:27.634247 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:27.634208    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"]
Jul 22 15:48:27.638396 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:27.638366    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-6f1c00-mggn7"]
Jul 22 15:48:28.601712 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:28.601680    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf226a03-aaed-413f-90c2-576b8ff2b0a3" path="/var/lib/kubelet/pods/bf226a03-aaed-413f-90c2-576b8ff2b0a3/volumes"
Jul 22 15:48:35.302830 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.302790    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"]
Jul 22 15:48:35.303229 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.303198    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2cd472f-17a8-4642-8760-4df286ef822b" containerName="maas-api"
Jul 22 15:48:35.303229 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.303211    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2cd472f-17a8-4642-8760-4df286ef822b" containerName="maas-api"
Jul 22 15:48:35.303229 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.303220    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf226a03-aaed-413f-90c2-576b8ff2b0a3" containerName="revoke-keys"
Jul 22 15:48:35.303229 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.303226    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf226a03-aaed-413f-90c2-576b8ff2b0a3" containerName="revoke-keys"
Jul 22 15:48:35.303378 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.303281    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2cd472f-17a8-4642-8760-4df286ef822b" containerName="maas-api"
Jul 22 15:48:35.303378 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.303293    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf226a03-aaed-413f-90c2-576b8ff2b0a3" containerName="revoke-keys"
Jul 22 15:48:35.308570 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.308539    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"
Jul 22 15:48:35.312392 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.312368    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b79tk\""
Jul 22 15:48:35.317659 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.317632    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"]
Jul 22 15:48:35.328431 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.328390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a528981b-f239-4595-a6d6-849ad810f4c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr\" (UID: \"a528981b-f239-4595-a6d6-849ad810f4c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"
Jul 22 15:48:35.429942 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.429889    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a528981b-f239-4595-a6d6-849ad810f4c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr\" (UID: \"a528981b-f239-4595-a6d6-849ad810f4c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"
Jul 22 15:48:35.430751 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.430724    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a528981b-f239-4595-a6d6-849ad810f4c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr\" (UID: \"a528981b-f239-4595-a6d6-849ad810f4c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"
Jul 22 15:48:35.620530 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.620422    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"
Jul 22 15:48:35.769967 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:35.769932    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"]
Jul 22 15:48:35.773005 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:48:35.772965    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda528981b_f239_4595_a6d6_849ad810f4c9.slice/crio-5678137ce249a63c5c1e3eab4f687a0a78bdfa1464a52aa524a1171d4d95113c WatchSource:0}: Error finding container 5678137ce249a63c5c1e3eab4f687a0a78bdfa1464a52aa524a1171d4d95113c: Status 404 returned error can't find the container with id 5678137ce249a63c5c1e3eab4f687a0a78bdfa1464a52aa524a1171d4d95113c
Jul 22 15:48:36.267119 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:36.267012    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr" event={"ID":"a528981b-f239-4595-a6d6-849ad810f4c9","Type":"ContainerStarted","Data":"9b4ae5342e4b7eb1a240c4efd72b3137fdb620f6ce17a14a35ffd381276fb80d"}
Jul 22 15:48:36.267119 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:36.267077    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr" event={"ID":"a528981b-f239-4595-a6d6-849ad810f4c9","Type":"ContainerStarted","Data":"5678137ce249a63c5c1e3eab4f687a0a78bdfa1464a52aa524a1171d4d95113c"}
Jul 22 15:48:36.285524 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:36.285466    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr" podStartSLOduration=1.285433035 podStartE2EDuration="1.285433035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:48:36.28399854 +0000 UTC m=+2654.738712154" watchObservedRunningTime="2026-07-22 15:48:36.285433035 +0000 UTC m=+2654.740146651"
Jul 22 15:48:37.273052 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:37.273016    2560 generic.go:356] "Generic (PLEG): container finished" podID="a528981b-f239-4595-a6d6-849ad810f4c9" containerID="9b4ae5342e4b7eb1a240c4efd72b3137fdb620f6ce17a14a35ffd381276fb80d" exitCode=0
Jul 22 15:48:37.273496 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:37.273073    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr" event={"ID":"a528981b-f239-4595-a6d6-849ad810f4c9","Type":"ContainerDied","Data":"9b4ae5342e4b7eb1a240c4efd72b3137fdb620f6ce17a14a35ffd381276fb80d"}
Jul 22 15:48:38.416874 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:38.416845    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"
Jul 22 15:48:38.455365 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:38.455330    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a528981b-f239-4595-a6d6-849ad810f4c9-maas-api-service-ca\") pod \"a528981b-f239-4595-a6d6-849ad810f4c9\" (UID: \"a528981b-f239-4595-a6d6-849ad810f4c9\") "
Jul 22 15:48:38.455711 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:38.455686    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a528981b-f239-4595-a6d6-849ad810f4c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a528981b-f239-4595-a6d6-849ad810f4c9" (UID: "a528981b-f239-4595-a6d6-849ad810f4c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:48:38.557123 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:38.557027    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a528981b-f239-4595-a6d6-849ad810f4c9-maas-api-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:39.284521 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:39.284482    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr" event={"ID":"a528981b-f239-4595-a6d6-849ad810f4c9","Type":"ContainerDied","Data":"5678137ce249a63c5c1e3eab4f687a0a78bdfa1464a52aa524a1171d4d95113c"}
Jul 22 15:48:39.284696 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:39.284568    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5678137ce249a63c5c1e3eab4f687a0a78bdfa1464a52aa524a1171d4d95113c"
Jul 22 15:48:39.284696 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:39.284501    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"
Jul 22 15:48:45.814100 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:45.814068    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"]
Jul 22 15:48:45.814510 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:45.814269    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl" podUID="f6ce32ef-d253-4998-b4af-249828380bf3" containerName="maas-api" containerID="cri-o://6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d" gracePeriod=30
Jul 22 15:48:45.843061 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:45.843030    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k"]
Jul 22 15:48:45.846340 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:45.846315    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-552a8b-29745585-2jc7k"]
Jul 22 15:48:46.061725 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.061699    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:48:46.129968 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.129878    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6ce32ef-d253-4998-b4af-249828380bf3-openshift-service-ca\") pod \"f6ce32ef-d253-4998-b4af-249828380bf3\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") "
Jul 22 15:48:46.130119 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.129976    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9f5f5\" (UniqueName: \"kubernetes.io/projected/f6ce32ef-d253-4998-b4af-249828380bf3-kube-api-access-9f5f5\") pod \"f6ce32ef-d253-4998-b4af-249828380bf3\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") "
Jul 22 15:48:46.130119 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.129998    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls\") pod \"f6ce32ef-d253-4998-b4af-249828380bf3\" (UID: \"f6ce32ef-d253-4998-b4af-249828380bf3\") "
Jul 22 15:48:46.130282 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.130248    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6ce32ef-d253-4998-b4af-249828380bf3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f6ce32ef-d253-4998-b4af-249828380bf3" (UID: "f6ce32ef-d253-4998-b4af-249828380bf3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 15:48:46.132143 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.132120    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f6ce32ef-d253-4998-b4af-249828380bf3" (UID: "f6ce32ef-d253-4998-b4af-249828380bf3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 15:48:46.132227 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.132181    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ce32ef-d253-4998-b4af-249828380bf3-kube-api-access-9f5f5" (OuterVolumeSpecName: "kube-api-access-9f5f5") pod "f6ce32ef-d253-4998-b4af-249828380bf3" (UID: "f6ce32ef-d253-4998-b4af-249828380bf3"). InnerVolumeSpecName "kube-api-access-9f5f5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 15:48:46.231732 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.231693    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6ce32ef-d253-4998-b4af-249828380bf3-openshift-service-ca\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:46.231732 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.231725    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f5f5\" (UniqueName: \"kubernetes.io/projected/f6ce32ef-d253-4998-b4af-249828380bf3-kube-api-access-9f5f5\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:46.231732 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.231740    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6ce32ef-d253-4998-b4af-249828380bf3-maas-api-tls\") on node \"ip-10-0-142-80.ec2.internal\" DevicePath \"\""
Jul 22 15:48:46.322914 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.322883    2560 generic.go:356] "Generic (PLEG): container finished" podID="f6ce32ef-d253-4998-b4af-249828380bf3" containerID="6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d" exitCode=0
Jul 22 15:48:46.323123 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.322930    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"
Jul 22 15:48:46.323123 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.322979    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl" event={"ID":"f6ce32ef-d253-4998-b4af-249828380bf3","Type":"ContainerDied","Data":"6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d"}
Jul 22 15:48:46.323123 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.323036    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl" event={"ID":"f6ce32ef-d253-4998-b4af-249828380bf3","Type":"ContainerDied","Data":"e22aa014f26e656e2e5a7021062c3cb3ae84887255eaea006f282ef22080fb46"}
Jul 22 15:48:46.323123 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.323065    2560 scope.go:117] "RemoveContainer" containerID="6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d"
Jul 22 15:48:46.338736 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.338714    2560 scope.go:117] "RemoveContainer" containerID="6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d"
Jul 22 15:48:46.339003 ip-10-0-142-80 kubenswrapper[2560]: E0722 15:48:46.338984    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d\": container with ID starting with 6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d not found: ID does not exist" containerID="6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d"
Jul 22 15:48:46.339057 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.339011    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d"} err="failed to get container status \"6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d\": rpc error: code = NotFound desc = could not find container \"6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d\": container with ID starting with 6ce7b5277a110ec02522a2451d10abd6b61c22a25b1eee0c14f31eb87a02263d not found: ID does not exist"
Jul 22 15:48:46.352884 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.352854    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"]
Jul 22 15:48:46.360108 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.360083    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-552a8b-654899579d-5qbtl"]
Jul 22 15:48:46.602625 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.602588    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e084579-dc9a-4a5e-9164-dd96daea2b59" path="/var/lib/kubelet/pods/2e084579-dc9a-4a5e-9164-dd96daea2b59/volumes"
Jul 22 15:48:46.602986 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:46.602973    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6ce32ef-d253-4998-b4af-249828380bf3" path="/var/lib/kubelet/pods/f6ce32ef-d253-4998-b4af-249828380bf3/volumes"
Jul 22 15:48:50.376484 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:50.376435    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"]
Jul 22 15:48:50.380136 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:50.380110    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-552a8b-bjqsr"]
Jul 22 15:48:50.602985 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:48:50.602947    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a528981b-f239-4595-a6d6-849ad810f4c9" path="/var/lib/kubelet/pods/a528981b-f239-4595-a6d6-849ad810f4c9/volumes"
Jul 22 15:49:04.825510 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:04.825472    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6995947675-wq9pq_527f7621-08df-413e-9c9c-7ee256afe0e6/manager/0.log"
Jul 22 15:49:05.198747 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:05.198715    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7777b6b677-k5hm4_12c27eb5-ef0d-4914-8325-7a9735960e92/copy-manifests/0.log"
Jul 22 15:49:05.216812 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:05.216775    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7777b6b677-k5hm4_12c27eb5-ef0d-4914-8325-7a9735960e92/manager/0.log"
Jul 22 15:49:06.119851 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:06.119817    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745570-cpmcm_25f7afb1-8597-4d0d-8ba7-e0c923a5194f/cleanup/0.log"
Jul 22 15:49:06.262250 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:06.262220    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745585-p989x_af6fc25b-587e-4465-a010-516bb4172c5e/cleanup/0.log"
Jul 22 15:49:06.385952 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:06.385855    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-jg2fb_78790f73-5d7a-4a33-bae8-d92dcee21bb1/postgres/0.log"
Jul 22 15:49:08.176693 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:08.176668    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zvfdm_30e63893-b387-47cd-b788-14b399146eed/kuadrant-console-plugin/0.log"
Jul 22 15:49:09.362083 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:09.362052    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59659ff488-g66vh_0c56b0d2-1712-4c40-9636-310afe140663/kube-auth-proxy/0.log"
Jul 22 15:49:09.760623 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:09.760592    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-4fldw_a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626/payload-pre-processing/0.log"
Jul 22 15:49:09.920780 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:09.920745    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wdvfv_6d0303e1-7bbf-4a02-bf7e-714766501070/payload-processing/0.log"
Jul 22 15:49:10.052182 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:10.052091    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d789dc46-blhfw_9e0637ad-06fc-496c-8a90-884a39cf26d9/router/0.log"
Jul 22 15:49:10.696641 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:10.696613    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-9mjwm_8849e49c-3323-4f58-b841-94ff6aba3eeb/main/0.log"
Jul 22 15:49:10.947511 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:10.947425    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-krrt8_cc6cdb3d-0269-4b19-8340-f6d2331b3717/main/0.log"
Jul 22 15:49:17.212917 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.212880    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6tddk/must-gather-7gq4d"]
Jul 22 15:49:17.213312 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.213261    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6ce32ef-d253-4998-b4af-249828380bf3" containerName="maas-api"
Jul 22 15:49:17.213312 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.213272    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6ce32ef-d253-4998-b4af-249828380bf3" containerName="maas-api"
Jul 22 15:49:17.213312 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.213286    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a528981b-f239-4595-a6d6-849ad810f4c9" containerName="revoke-keys"
Jul 22 15:49:17.213312 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.213294    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a528981b-f239-4595-a6d6-849ad810f4c9" containerName="revoke-keys"
Jul 22 15:49:17.213440 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.213363    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6ce32ef-d253-4998-b4af-249828380bf3" containerName="maas-api"
Jul 22 15:49:17.213440 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.213374    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a528981b-f239-4595-a6d6-849ad810f4c9" containerName="revoke-keys"
Jul 22 15:49:17.216592 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.216572    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tddk/must-gather-7gq4d"
Jul 22 15:49:17.231766 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.231742    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tddk\"/\"openshift-service-ca.crt\""
Jul 22 15:49:17.232182 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.232160    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6tddk\"/\"default-dockercfg-9pgbb\""
Jul 22 15:49:17.232242 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.232225    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tddk\"/\"kube-root-ca.crt\""
Jul 22 15:49:17.261787 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.261757    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tddk/must-gather-7gq4d"]
Jul 22 15:49:17.312541 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.312500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmcql\" (UniqueName: \"kubernetes.io/projected/eb113bbb-d6a5-45a3-b092-576cfd61eb91-kube-api-access-bmcql\") pod \"must-gather-7gq4d\" (UID: \"eb113bbb-d6a5-45a3-b092-576cfd61eb91\") " pod="openshift-must-gather-6tddk/must-gather-7gq4d"
Jul 22 15:49:17.312745 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.312571    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eb113bbb-d6a5-45a3-b092-576cfd61eb91-must-gather-output\") pod \"must-gather-7gq4d\" (UID: \"eb113bbb-d6a5-45a3-b092-576cfd61eb91\") " pod="openshift-must-gather-6tddk/must-gather-7gq4d"
Jul 22 15:49:17.413542 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.413508    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eb113bbb-d6a5-45a3-b092-576cfd61eb91-must-gather-output\") pod \"must-gather-7gq4d\" (UID: \"eb113bbb-d6a5-45a3-b092-576cfd61eb91\") " pod="openshift-must-gather-6tddk/must-gather-7gq4d"
Jul 22 15:49:17.413720 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.413585    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmcql\" (UniqueName: \"kubernetes.io/projected/eb113bbb-d6a5-45a3-b092-576cfd61eb91-kube-api-access-bmcql\") pod \"must-gather-7gq4d\" (UID: \"eb113bbb-d6a5-45a3-b092-576cfd61eb91\") " pod="openshift-must-gather-6tddk/must-gather-7gq4d"
Jul 22 15:49:17.413854 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.413835    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eb113bbb-d6a5-45a3-b092-576cfd61eb91-must-gather-output\") pod \"must-gather-7gq4d\" (UID: \"eb113bbb-d6a5-45a3-b092-576cfd61eb91\") " pod="openshift-must-gather-6tddk/must-gather-7gq4d"
Jul 22 15:49:17.433613 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.433582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmcql\" (UniqueName: \"kubernetes.io/projected/eb113bbb-d6a5-45a3-b092-576cfd61eb91-kube-api-access-bmcql\") pod \"must-gather-7gq4d\" (UID: \"eb113bbb-d6a5-45a3-b092-576cfd61eb91\") " pod="openshift-must-gather-6tddk/must-gather-7gq4d"
Jul 22 15:49:17.525645 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.525542    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tddk/must-gather-7gq4d"
Jul 22 15:49:17.667659 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:17.667630    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tddk/must-gather-7gq4d"]
Jul 22 15:49:17.671493 ip-10-0-142-80 kubenswrapper[2560]: W0722 15:49:17.671463    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb113bbb_d6a5_45a3_b092_576cfd61eb91.slice/crio-3821be337c05f3bdb30ee2e9c0a2a06d907346b234b1ff5f1948536d8af85c33 WatchSource:0}: Error finding container 3821be337c05f3bdb30ee2e9c0a2a06d907346b234b1ff5f1948536d8af85c33: Status 404 returned error can't find the container with id 3821be337c05f3bdb30ee2e9c0a2a06d907346b234b1ff5f1948536d8af85c33
Jul 22 15:49:18.488064 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:18.487966    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tddk/must-gather-7gq4d" event={"ID":"eb113bbb-d6a5-45a3-b092-576cfd61eb91","Type":"ContainerStarted","Data":"2345e402962a6fd582840a0dd80ac847acb84792fb00a4df3a577b0093cf9c64"}
Jul 22 15:49:18.488064 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:18.488027    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tddk/must-gather-7gq4d" event={"ID":"eb113bbb-d6a5-45a3-b092-576cfd61eb91","Type":"ContainerStarted","Data":"3821be337c05f3bdb30ee2e9c0a2a06d907346b234b1ff5f1948536d8af85c33"}
Jul 22 15:49:19.499834 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:19.499790    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tddk/must-gather-7gq4d" event={"ID":"eb113bbb-d6a5-45a3-b092-576cfd61eb91","Type":"ContainerStarted","Data":"59ef8065e3084ae213dc1e0dd4f70413336292bb4ff0ca8bf5e11059369481c5"}
Jul 22 15:49:19.518062 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:19.517994    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6tddk/must-gather-7gq4d" podStartSLOduration=1.832679251 podStartE2EDuration="2.51795501s" totalImagesPullingTime="685.275759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 15:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:49:19.51744235 +0000 UTC m=+2697.972155967" watchObservedRunningTime="2026-07-22 15:49:19.51795501 +0000 UTC m=+2697.972668891"
Jul 22 15:49:20.042432 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:20.042401    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c8jkn_2b0d8426-5e95-49b9-b057-b5e5cfc59182/global-pull-secret-syncer/0.log"
Jul 22 15:49:20.195245 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:20.195206    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4nbnn_12e87e9c-0d5e-4be3-99be-110478935005/konnectivity-agent/0.log"
Jul 22 15:49:20.304878 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:20.304784    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-80.ec2.internal_ddc4a6bf9a8830c582e31a32617937e6/haproxy/0.log"
Jul 22 15:49:22.302515 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:22.301842    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:49:22.309465 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:22.308196    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:49:22.338467 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:22.337583    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:49:22.338467 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:22.338393    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf658_9d0d4357-a4b6-43fb-8587-4b50695f40e5/ovn-acl-logging/0.log"
Jul 22 15:49:25.177139 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:25.177009    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zvfdm_30e63893-b387-47cd-b788-14b399146eed/kuadrant-console-plugin/0.log"
Jul 22 15:49:27.010057 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.009431    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7096c167-bdc7-408b-acea-15439fe7b559/alertmanager/0.log"
Jul 22 15:49:27.041831 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.041796    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7096c167-bdc7-408b-acea-15439fe7b559/config-reloader/0.log"
Jul 22 15:49:27.080124 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.080042    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7096c167-bdc7-408b-acea-15439fe7b559/kube-rbac-proxy-web/0.log"
Jul 22 15:49:27.111342 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.111259    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7096c167-bdc7-408b-acea-15439fe7b559/kube-rbac-proxy/0.log"
Jul 22 15:49:27.142377 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.142352    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7096c167-bdc7-408b-acea-15439fe7b559/kube-rbac-proxy-metric/0.log"
Jul 22 15:49:27.172302 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.172217    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7096c167-bdc7-408b-acea-15439fe7b559/prom-label-proxy/0.log"
Jul 22 15:49:27.198315 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.198233    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7096c167-bdc7-408b-acea-15439fe7b559/init-config-reloader/0.log"
Jul 22 15:49:27.803437 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.803401    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tvwsg_251ab756-226a-491c-ab81-d021165a4738/node-exporter/0.log"
Jul 22 15:49:27.828225 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.828201    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tvwsg_251ab756-226a-491c-ab81-d021165a4738/kube-rbac-proxy/0.log"
Jul 22 15:49:27.852258 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.852228    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tvwsg_251ab756-226a-491c-ab81-d021165a4738/init-textfile/0.log"
Jul 22 15:49:27.967847 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.967815    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea/prometheus/0.log"
Jul 22 15:49:27.993413 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:27.993383    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea/config-reloader/0.log"
Jul 22 15:49:28.023476 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.023358    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea/thanos-sidecar/0.log"
Jul 22 15:49:28.052019 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.051979    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea/kube-rbac-proxy-web/0.log"
Jul 22 15:49:28.081104 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.081056    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea/kube-rbac-proxy/0.log"
Jul 22 15:49:28.106466 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.106186    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea/kube-rbac-proxy-thanos/0.log"
Jul 22 15:49:28.132442 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.132186    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa8eb493-f4b1-4be7-8311-7c59d3e9a2ea/init-config-reloader/0.log"
Jul 22 15:49:28.335283 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.335192    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-db8557d74-dcchm_d00bf790-d0db-4f65-9a4d-2d17f7edd1db/thanos-query/0.log"
Jul 22 15:49:28.358260 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.358230    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-db8557d74-dcchm_d00bf790-d0db-4f65-9a4d-2d17f7edd1db/kube-rbac-proxy-web/0.log"
Jul 22 15:49:28.381007 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.380969    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-db8557d74-dcchm_d00bf790-d0db-4f65-9a4d-2d17f7edd1db/kube-rbac-proxy/0.log"
Jul 22 15:49:28.402441 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.402410    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-db8557d74-dcchm_d00bf790-d0db-4f65-9a4d-2d17f7edd1db/prom-label-proxy/0.log"
Jul 22 15:49:28.424831 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.424793    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-db8557d74-dcchm_d00bf790-d0db-4f65-9a4d-2d17f7edd1db/kube-rbac-proxy-rules/0.log"
Jul 22 15:49:28.448925 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:28.448895    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-db8557d74-dcchm_d00bf790-d0db-4f65-9a4d-2d17f7edd1db/kube-rbac-proxy-metrics/0.log"
Jul 22 15:49:30.200877 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.200833    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"]
Jul 22 15:49:30.205881 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.205849    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.216498 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.216459    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"]
Jul 22 15:49:30.251240 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.251208    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-lib-modules\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.251525 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.251508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-podres\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.251683 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.251667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4669\" (UniqueName: \"kubernetes.io/projected/8f88156b-3136-4fdc-ab9c-1900977a32b3-kube-api-access-n4669\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.251847 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.251834    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-proc\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.251990 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.251978    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-sys\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.311368 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.311342    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/1.log"
Jul 22 15:49:30.321498 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.321442    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-mnzqv_d1269636-75f4-4267-b9d6-302fd715c279/console-operator/2.log"
Jul 22 15:49:30.353611 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.353567    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-lib-modules\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.353802 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.353629    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-podres\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.353802 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.353683    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4669\" (UniqueName: \"kubernetes.io/projected/8f88156b-3136-4fdc-ab9c-1900977a32b3-kube-api-access-n4669\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.353802 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.353737    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-proc\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.353802 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.353777    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-lib-modules\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.354025 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.353777    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-podres\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.354025 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.353783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-sys\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.354025 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.353834    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-proc\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.354025 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.353954    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f88156b-3136-4fdc-ab9c-1900977a32b3-sys\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.363848 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.363805    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4669\" (UniqueName: \"kubernetes.io/projected/8f88156b-3136-4fdc-ab9c-1900977a32b3-kube-api-access-n4669\") pod \"perf-node-gather-daemonset-th5h4\" (UID: \"8f88156b-3136-4fdc-ab9c-1900977a32b3\") " pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.523602 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.523508    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:30.712734 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.712697    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"]
Jul 22 15:49:30.975040 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:30.975006    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-pxqhr_41fe00d2-2f54-4e8f-b445-91a5e4da5936/download-server/0.log"
Jul 22 15:49:31.605552 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:31.605514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4" event={"ID":"8f88156b-3136-4fdc-ab9c-1900977a32b3","Type":"ContainerStarted","Data":"64fde9c150a49f5c2c0528c1afa993fe1c773c544c52c54bc95a4080bf1ef910"}
Jul 22 15:49:31.606126 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:31.605585    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4" event={"ID":"8f88156b-3136-4fdc-ab9c-1900977a32b3","Type":"ContainerStarted","Data":"d8347c139e84ae5b30bd921e9dda827acfbda61cc04ac595f5e3d485d33b494e"}
Jul 22 15:49:31.606126 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:31.605838    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:31.626281 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:31.626214    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4" podStartSLOduration=1.626190309 podStartE2EDuration="1.626190309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 15:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 15:49:31.624783899 +0000 UTC m=+2710.079497507" watchObservedRunningTime="2026-07-22 15:49:31.626190309 +0000 UTC m=+2710.080903925"
Jul 22 15:49:32.670425 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:32.670395    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pblnf_08cc6e66-2c7b-483d-9d70-b96de5dd2c7a/dns/0.log"
Jul 22 15:49:32.691200 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:32.691171    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pblnf_08cc6e66-2c7b-483d-9d70-b96de5dd2c7a/kube-rbac-proxy/0.log"
Jul 22 15:49:32.857027 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:32.856997    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z4765_dd87a8d8-cc8e-4b2c-82c7-a9b81a5c016f/dns-node-resolver/0.log"
Jul 22 15:49:33.487929 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:33.487894    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hkpl4_3d1e6293-7310-4977-81ce-f804acdcbcd0/node-ca/0.log"
Jul 22 15:49:34.616725 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:34.616697    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59659ff488-g66vh_0c56b0d2-1712-4c40-9636-310afe140663/kube-auth-proxy/0.log"
Jul 22 15:49:34.742683 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:34.742638    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-4fldw_a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626/payload-pre-processing/0.log"
Jul 22 15:49:34.785671 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:34.785634    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wdvfv_6d0303e1-7bbf-4a02-bf7e-714766501070/payload-processing/0.log"
Jul 22 15:49:34.814427 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:34.814397    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d789dc46-blhfw_9e0637ad-06fc-496c-8a90-884a39cf26d9/router/0.log"
Jul 22 15:49:35.591879 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:35.591847    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gmpdw_6c583dee-6f25-4c83-9858-da1763f2a5f5/serve-healthcheck-canary/0.log"
Jul 22 15:49:36.196245 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:36.196206    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pbrrl_b713f6e2-248e-4563-b2c3-682c1fc41469/kube-rbac-proxy/0.log"
Jul 22 15:49:36.217952 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:36.217926    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pbrrl_b713f6e2-248e-4563-b2c3-682c1fc41469/exporter/0.log"
Jul 22 15:49:36.238388 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:36.238359    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pbrrl_b713f6e2-248e-4563-b2c3-682c1fc41469/extractor/0.log"
Jul 22 15:49:37.623960 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:37.622949    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6tddk/perf-node-gather-daemonset-th5h4"
Jul 22 15:49:38.891930 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:38.891888    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6995947675-wq9pq_527f7621-08df-413e-9c9c-7ee256afe0e6/manager/0.log"
Jul 22 15:49:39.136556 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:39.136300    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7777b6b677-k5hm4_12c27eb5-ef0d-4914-8325-7a9735960e92/manager/0.log"
Jul 22 15:49:39.155669 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:39.155637    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7777b6b677-k5hm4_12c27eb5-ef0d-4914-8325-7a9735960e92/copy-manifests/0.log"
Jul 22 15:49:40.658680 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:40.658647    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-79668d66cc-pxqtr_8d86ce5a-a82b-4fdb-bae5-6ba4c44eec35/manager/0.log"
Jul 22 15:49:41.172062 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:41.172036    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59659ff488-g66vh_0c56b0d2-1712-4c40-9636-310afe140663/kube-auth-proxy/0.log"
Jul 22 15:49:41.272219 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:41.272185    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-4fldw_a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626/payload-pre-processing/0.log"
Jul 22 15:49:41.302768 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:41.302739    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wdvfv_6d0303e1-7bbf-4a02-bf7e-714766501070/payload-processing/0.log"
Jul 22 15:49:41.317048 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:41.317023    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d789dc46-blhfw_9e0637ad-06fc-496c-8a90-884a39cf26d9/router/0.log"
Jul 22 15:49:42.629733 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:42.629646    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59659ff488-g66vh_0c56b0d2-1712-4c40-9636-310afe140663/kube-auth-proxy/0.log"
Jul 22 15:49:42.727472 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:42.727417    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-4fldw_a99ae2b0-eb0b-4a6f-9ced-f7b6acc63626/payload-pre-processing/0.log"
Jul 22 15:49:42.756859 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:42.756824    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wdvfv_6d0303e1-7bbf-4a02-bf7e-714766501070/payload-processing/0.log"
Jul 22 15:49:42.775291 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:42.775263    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d789dc46-blhfw_9e0637ad-06fc-496c-8a90-884a39cf26d9/router/0.log"
Jul 22 15:49:47.901922 ip-10-0-142-80 kubenswrapper[2560]: I0722 15:49:47.901875    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-45g86_871c5594-a183-4116-8d8a-20ee028153f5/kube-multus/0.log"