Jul 24 03:15:16.236441 ip-10-0-132-49 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 03:15:16.236452 ip-10-0-132-49 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 03:15:16.236462 ip-10-0-132-49 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 03:15:16.236774 ip-10-0-132-49 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 03:15:26.426262 ip-10-0-132-49 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 03:15:26.426283 ip-10-0-132-49 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 439aca25ba53428fb999556338af0fca --
Jul 24 03:17:11.197713 ip-10-0-132-49 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 03:17:11.501312 ip-10-0-132-49 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 03:17:11.501312 ip-10-0-132-49 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 03:17:11.501312 ip-10-0-132-49 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 03:17:11.501312 ip-10-0-132-49 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 03:17:11.501312 ip-10-0-132-49 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 03:17:11.503287 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.503157    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 03:17:11.509297 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509264    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:11.509297 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509286    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:11.509297 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509290    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:11.509297 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509293    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:11.509297 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509296    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:11.509297 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509300    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:11.509297 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509303    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:11.509297 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509307    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509310    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509314    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509318    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509322    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509325    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509328    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509331    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509334    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509337    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509341    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509344    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509347    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509350    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509352    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509355    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509358    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509360    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:11.509607 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509363    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509365    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509368    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509370    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509373    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509375    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509394    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509404    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509407    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509411    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509414    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509416    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509419    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509421    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509424    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509427    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509430    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509433    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509436    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509438    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:11.510037 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509441    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509444    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509446    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509449    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509451    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509454    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509456    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509459    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509462    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509464    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509466    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509469    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509471    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509474    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509476    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509479    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509481    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509484    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509486    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509489    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:11.510536 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509491    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509494    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509497    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509500    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509502    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509505    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509507    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509511    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509514    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509516    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509519    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509521    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509524    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509526    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509529    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509531    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509533    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509536    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509539    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509541    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:11.511015 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509544    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509547    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509549    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509551    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509554    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509556    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509559    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509561    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509563    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509566    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509568    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509572    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509575    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.509578    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510904    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510912    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510916    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510919    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510926    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:11.511497 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510931    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510934    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510938    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510940    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510943    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510946    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510949    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510952    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510954    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510957    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510959    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510961    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510965    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510967    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510970    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510972    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510975    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510978    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510980    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510983    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:11.511979 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510985    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510988    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510990    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510992    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510995    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.510997    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511000    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511002    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511005    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511007    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511010    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511014    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511017    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511020    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511023    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511026    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511028    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511031    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511034    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511036    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:11.512617 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511039    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511042    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511045    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511047    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511050    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511053    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511055    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511058    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511060    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511062    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511065    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511067    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511069    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511073    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511075    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511077    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511080    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511082    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511085    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511087    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:11.513106 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511091    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511094    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511097    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511101    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511103    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511107    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511109    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511112    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511114    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511117    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511119    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511122    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511124    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511126    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511129    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511131    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511134    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511136    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511139    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511141    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:11.513614 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511144    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511146    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511148    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511151    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511156    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511159    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511161    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511164    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511167    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511169    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511172    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511175    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511177    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511179    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511627    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511633    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511636    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511642    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511646    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:11.514099 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511649    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511652    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511655    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511658    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511661    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511663    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511666    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511669    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511671    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511674    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511676    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511679    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511681    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511684    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511687    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511690    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511692    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511695    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511697    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511701    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:11.514601 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511705    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511708    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511711    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511713    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511715    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511718    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511720    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511723    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511726    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511729    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511732    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511735    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511737    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511740    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511742    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511744    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511747    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511749    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511752    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511754    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511756    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:11.515104 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511759    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511762    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511764    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511767    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511770    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511772    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511774    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511777    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511779    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511781    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511784    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511786    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511789    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511791    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511794    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511796    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511798    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511802    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511805    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:11.515659 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511808    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511811    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511815    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511819    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511822    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511824    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511827    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511829    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511832    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511834    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511836    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511839    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511841    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511844    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511846    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511849    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511851    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511854    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511857    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511859    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:11.516180 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511861    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511864    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511866    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511869    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511871    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511882    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511884    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511886    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511889    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511891    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511896    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511898    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511901    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.511904    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.511981    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.511990    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.511997    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512001    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512005    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512008    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 03:17:11.516687 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512012    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512017    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512020    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512023    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512027    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512030    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512033    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512037    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512039    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512042    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512045    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512048    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512053    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512055    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512058    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512061    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512064    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512068    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512071    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512074    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512077    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512080    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512084    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512087    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 03:17:11.517168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512090    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512093    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512097    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512100    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512103    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512105    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512108    2577 flags.go:64] FLAG: --enable-server="true"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512111    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512116    2577 flags.go:64] FLAG: --event-burst="100"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512119    2577 flags.go:64] FLAG: --event-qps="50"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512122    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512125    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512127    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512136    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512139    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512142    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512144    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512160    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512164    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512168    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512172    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512175    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512178    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512181    2577 flags.go:64] FLAG: --feature-gates=""
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512185    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 03:17:11.517787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512188    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512191    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512194    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512197    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512200    2577 flags.go:64] FLAG: --help="false"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512204    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512207    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512210    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512213    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512216    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512219    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512222    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512225    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512228    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512230    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512233    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512236    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512238    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512241    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512244    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512246    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512249    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512252    2577 flags.go:64] FLAG: --lock-file=""
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512255    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 03:17:11.518432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512257    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512260    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512266    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512269    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512272    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512274    2577 flags.go:64] FLAG: --logging-format="text"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512277    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512281    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512283    2577 flags.go:64] FLAG: --manifest-url=""
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512286    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512290    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512293    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512297    2577 flags.go:64] FLAG: --max-pods="110"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512302    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512304    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512307    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512310    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512313    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512316    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512319    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512327    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512330    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512333    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 03:17:11.519020 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512335    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512338    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512344    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512347    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512350    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512353    2577 flags.go:64] FLAG: --port="10250"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512356    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512358    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b4236b5d31ebd904"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512361    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512364    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512367    2577 flags.go:64] FLAG: --register-node="true"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512370    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512373    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512394    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512399    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512402    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512405    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512408    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512411    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512414    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512417    2577 flags.go:64] FLAG: --runonce="false"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512420    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512423    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512427    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512430    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 03:17:11.519591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512434    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512437    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512440    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512443    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512446    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512449    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512452    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512454    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512457    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512460    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512463    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512468    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512476    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512478    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512482    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512485    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512488    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512490    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512493    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512496    2577 flags.go:64] FLAG: --v="2"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512500    2577 flags.go:64] FLAG: --version="false"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512510    2577 flags.go:64] FLAG: --vmodule=""
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512513    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512516    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 03:17:11.520206 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512638    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512643    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512647    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512650    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512653    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512656    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512660    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512664    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512667    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512670    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512673    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512676    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512678    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512681    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512683    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512686    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512689    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512691    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512694    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:11.520851 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512696    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512698    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512701    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512703    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512706    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512709    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512713    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512716    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512718    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512721    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512724    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512727    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512729    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512732    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512735    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512737    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512739    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512742    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512744    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:11.521310 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512750    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512753    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512756    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512758    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512761    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512763    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512766    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512768    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512772    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512775    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512778    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512781    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512784    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512787    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512789    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512791    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512794    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512797    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512799    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:11.521785 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512801    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512804    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512807    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512809    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512811    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512814    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512817    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512819    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512822    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512824    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512826    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512829    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512832    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512836    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512840    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512843    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512845    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512848    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512850    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512853    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:11.522245 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512855    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512858    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512860    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512863    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512865    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512868    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512870    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512873    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512875    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512877    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512880    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512882    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512884    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512887    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512889    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512892    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512894    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512897    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512899    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512902    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:11.522764 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512905    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.512907    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.512913    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.519698    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.519714    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.519735    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.519742    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519802    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519806    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519809    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519812    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519815    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519818    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519821    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519824    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:11.523256 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519827    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519830    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519833    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519836    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519839    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519842    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519845    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519847    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519850    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519852    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519854    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519857    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519859    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519862    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519865    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519867    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519870    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519882    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519886    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:11.523700 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519891    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519895    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519899    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519902    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519904    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519907    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519909    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519912    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519915    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519918    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519920    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519923    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519925    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519927    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519930    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519932    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519935    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519937    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519940    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519943    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:11.524164 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519945    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519947    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519950    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519952    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519955    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519957    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519960    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519962    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519965    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519968    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519972    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519975    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519978    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519981    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519984    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519986    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519989    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519991    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519994    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:11.524677 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519996    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.519999    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520001    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520004    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520006    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520008    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520011    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520013    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520016    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520018    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520021    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520023    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520026    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520028    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520031    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520033    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520035    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520038    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520040    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520043    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:11.525147 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520045    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520048    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520050    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520053    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520057    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520059    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520062    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520065    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520067    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520070    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520072    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520075    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520077    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.520082    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520195    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:11.525647 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520199    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520202    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520205    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520207    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520210    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520213    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520216    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520219    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520221    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520224    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520227    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520229    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520232    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520234    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520236    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520239    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520241    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520244    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520246    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520248    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:11.526025 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520251    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520253    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520256    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520259    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520262    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520264    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520267    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520269    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520272    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520274    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520277    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520279    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520282    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520286    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520289    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520291    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520294    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520296    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520298    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:11.526531 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520301    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520303    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520305    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520308    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520310    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520313    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520315    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520318    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520320    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520323    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520325    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520327    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520330    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520333    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520336    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520338    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520341    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520344    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520346    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520349    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:11.526990 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520351    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520353    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520356    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520358    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520360    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520363    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520365    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520367    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520370    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520372    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520374    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520397    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520400    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520403    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520405    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520408    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520410    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520413    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520416    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520418    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:11.527483 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520422    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520425    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520428    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520431    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520433    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520437    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520439    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520442    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520444    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520448    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520451    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520453    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520456    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520458    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520461    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520463    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520466    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520470    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:11.527975 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:11.520473    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:11.528574 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.520479    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 03:17:11.528574 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.520601    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 03:17:11.528574 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.522659    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 03:17:11.528574 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.523731    2577 server.go:1019] "Starting client certificate rotation"
Jul 24 03:17:11.528574 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.523836    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 03:17:11.528574 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.523881    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 03:17:11.528574 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.525767    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 03:17:11.528963 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.528946    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 03:17:11.536908 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.536888    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 24 03:17:11.540645 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.540625    2577 log.go:25] "Validated CRI v1 image API"
Jul 24 03:17:11.543109 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.543092    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 03:17:11.546496 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.546472    2577 fs.go:135] Filesystem UUIDs: map[7016c84b-9b3d-4116-a355-d15e4fc9126d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c09c5d29-d53e-460e-959c-a865f73fcade:/dev/nvme0n1p3]
Jul 24 03:17:11.546571 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.546495    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 03:17:11.551742 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.551615    2577 manager.go:216] Machine: {Timestamp:2026-07-24 03:17:11.550348314 +0000 UTC m=+0.260140293 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097167 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2509048c3ef1e762a9a56731c6c067 SystemUUID:ec250904-8c3e-f1e7-62a9-a56731c6c067 BootID:439aca25-ba53-428f-b999-556338af0fca Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:81:db:b7:fd:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:81:db:b7:fd:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:13:52:d5:ec:40 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 03:17:11.551742 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.551729    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 03:17:11.551905 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.551814    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 03:17:11.552061 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.552045    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 03:17:11.554369 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.554343    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 03:17:11.554521 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.554373    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-49.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 03:17:11.555406 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.555396    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 03:17:11.555452 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.555409    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 03:17:11.555452 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.555417    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 03:17:11.555452 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.555428    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 03:17:11.556224 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.556214    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 03:17:11.558904 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.558890    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 03:17:11.559084 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.559074    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 03:17:11.559115 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.559106    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 03:17:11.559143 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.559116    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 03:17:11.559143 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.559131    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 03:17:11.559143 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.559142    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 24 03:17:11.559220 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.559154    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 03:17:11.559619 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.559607    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 03:17:11.559681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.559630    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 03:17:11.561449 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.561429    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 03:17:11.563245 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.563231    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 03:17:11.563309 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.563262    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 03:17:11.564728 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564711    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 03:17:11.564728 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564729    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 03:17:11.564728 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564735    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 03:17:11.564898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564740    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 03:17:11.564898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564749    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 03:17:11.564898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564755    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 03:17:11.564898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564762    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 03:17:11.564898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564768    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 03:17:11.564898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564775    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 03:17:11.564898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564781    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 03:17:11.564898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564789    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 03:17:11.564898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.564795    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 03:17:11.565347 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.565336    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 03:17:11.565347 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.565345    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 03:17:11.568562 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.568548    2577 server.go:1268] "Started kubelet"
Jul 24 03:17:11.568770 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.568716    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 03:17:11.568863 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.568810    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 03:17:11.571399 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.571356    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 03:17:11.572177 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.572146    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 03:17:11.572247 ip-10-0-132-49 systemd[1]: Started Kubernetes Kubelet.
Jul 24 03:17:11.572641 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.572610    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-49.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 03:17:11.574909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.572900    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-49.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 03:17:11.574909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.573099    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 03:17:11.574909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.574418    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 03:17:11.575192 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.575168    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 03:17:11.577543 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.576806    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-49.ec2.internal.18c51b78795e4bad  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-49.ec2.internal,UID:ip-10-0-132-49.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-49.ec2.internal,},FirstTimestamp:2026-07-24 03:17:11.568518061 +0000 UTC m=+0.278310041,LastTimestamp:2026-07-24 03:17:11.568518061 +0000 UTC m=+0.278310041,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-49.ec2.internal,}"
Jul 24 03:17:11.581569 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.581551    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 03:17:11.581569 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.581559    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 03:17:11.581790 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.581771    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:11.582257 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582233    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 03:17:11.582257 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582234    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 03:17:11.582425 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582265    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 03:17:11.582425 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582349    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 03:17:11.582425 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582358    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 03:17:11.582565 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582428    2577 factory.go:55] Registering systemd factory
Jul 24 03:17:11.582565 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582464    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 24 03:17:11.582748 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582731    2577 factory.go:153] Registering CRI-O factory
Jul 24 03:17:11.582748 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582749    2577 factory.go:223] Registration of the crio container factory successfully
Jul 24 03:17:11.582900 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582798    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 03:17:11.582900 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582823    2577 factory.go:103] Registering Raw factory
Jul 24 03:17:11.582900 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.582838    2577 manager.go:1196] Started watching for new ooms in manager
Jul 24 03:17:11.583706 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.583690    2577 manager.go:349] Starting recovery of all containers
Jul 24 03:17:11.593371 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.593349    2577 manager.go:354] Recovery completed
Jul 24 03:17:11.595064 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.595036    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 03:17:11.595232 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.595044    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-49.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 03:17:11.598978 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.598963    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:11.601042 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.600907    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:11.601112 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.601058    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:11.601112 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.601072    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:11.601701 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.601686    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 03:17:11.601767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.601704    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 03:17:11.601767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.601724    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 03:17:11.601821 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.601695    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:11.602584 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.602548    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l42dw"
Jul 24 03:17:11.602646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.602630    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:11.602729 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.602651    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:11.602729 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.602662    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:11.604297 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.604284    2577 policy_none.go:49] "None policy: Start"
Jul 24 03:17:11.604358 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.604301    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 03:17:11.604358 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.604312    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 03:17:11.606306 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.606294    2577 policy_none.go:47] "Start"
Jul 24 03:17:11.608870 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.608847    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l42dw"
Jul 24 03:17:11.610893 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.610808    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-49.ec2.internal.18c51b787b4e9068  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-49.ec2.internal,UID:ip-10-0-132-49.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-49.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-49.ec2.internal,},FirstTimestamp:2026-07-24 03:17:11.601041512 +0000 UTC m=+0.310833494,LastTimestamp:2026-07-24 03:17:11.601041512 +0000 UTC m=+0.310833494,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-49.ec2.internal,}"
Jul 24 03:17:11.614596 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.614532    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-49.ec2.internal.18c51b787b4eefdb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-49.ec2.internal,UID:ip-10-0-132-49.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-49.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-49.ec2.internal,},FirstTimestamp:2026-07-24 03:17:11.601065947 +0000 UTC m=+0.310857927,LastTimestamp:2026-07-24 03:17:11.601065947 +0000 UTC m=+0.310857927,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-49.ec2.internal,}"
Jul 24 03:17:11.638956 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.638936    2577 manager.go:337] "Starting Device Plugin manager"
Jul 24 03:17:11.639065 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.638972    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 03:17:11.639065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.638983    2577 server.go:87] "Starting device plugin registration server"
Jul 24 03:17:11.639291 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.639277    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 03:17:11.639325 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.639296    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 03:17:11.639404 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.639369    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 03:17:11.639486 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.639473    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 03:17:11.639486 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.639485    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 03:17:11.640069 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.640050    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 03:17:11.640151 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.640105    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:11.685893 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.685858    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 03:17:11.692776 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.687164    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 03:17:11.692776 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.687198    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 03:17:11.692776 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.687225    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 03:17:11.692776 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.687260    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 03:17:11.692776 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.689628    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:11.739581 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.739549    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:11.740620 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.740601    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:11.740720 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.740635    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:11.740720 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.740646    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:11.740720 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.740672    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.749874 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.749850    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.749972 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.749878    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-49.ec2.internal\": node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:11.777913 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.777829    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:11.788089 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.788048    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"]
Jul 24 03:17:11.788213 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.788144    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:11.789098 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.789081    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:11.789196 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.789117    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:11.789196 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.789131    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:11.791547 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.791530    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:11.791664 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.791648    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.791729 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.791683    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:11.792292 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.792268    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:11.792401 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.792268    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:11.792401 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.792328    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:11.792401 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.792349    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:11.792401 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.792301    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:11.792401 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.792395    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:11.794685 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.794668    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.794760 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.794709    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:11.795441 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.795426    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:11.795493 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.795462    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:11.795493 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.795480    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:11.818228 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.818203    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-49.ec2.internal\" not found" node="ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.822593 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.822575    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-49.ec2.internal\" not found" node="ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.878324 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.878286    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:11.883634 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.883610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a4fbcf9cbb15b17f76420beb7ed4f29-config\") pod \"kube-apiserver-proxy-ip-10-0-132-49.ec2.internal\" (UID: \"3a4fbcf9cbb15b17f76420beb7ed4f29\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.883735 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.883646    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a261e7af2cd24409d94213307b3a2dcd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"a261e7af2cd24409d94213307b3a2dcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.883735 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.883673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a261e7af2cd24409d94213307b3a2dcd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"a261e7af2cd24409d94213307b3a2dcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.979167 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:11.979125    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:11.984516 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.984490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a261e7af2cd24409d94213307b3a2dcd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"a261e7af2cd24409d94213307b3a2dcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.984590 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.984523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a4fbcf9cbb15b17f76420beb7ed4f29-config\") pod \"kube-apiserver-proxy-ip-10-0-132-49.ec2.internal\" (UID: \"3a4fbcf9cbb15b17f76420beb7ed4f29\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.984590 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.984544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a261e7af2cd24409d94213307b3a2dcd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"a261e7af2cd24409d94213307b3a2dcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.984590 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.984588    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a261e7af2cd24409d94213307b3a2dcd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"a261e7af2cd24409d94213307b3a2dcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.984728 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.984596    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a4fbcf9cbb15b17f76420beb7ed4f29-config\") pod \"kube-apiserver-proxy-ip-10-0-132-49.ec2.internal\" (UID: \"3a4fbcf9cbb15b17f76420beb7ed4f29\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:11.984728 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:11.984594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a261e7af2cd24409d94213307b3a2dcd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal\" (UID: \"a261e7af2cd24409d94213307b3a2dcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:12.080033 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:12.079947    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:12.120575 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.120542    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:12.126523 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.126495    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:12.180339 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:12.180300    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:12.280854 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:12.280808    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:12.381405 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:12.381311    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-49.ec2.internal\" not found"
Jul 24 03:17:12.413233 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.413203    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:12.428213 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.428188    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:12.482059 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.482023    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:12.497848 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.497816    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 03:17:12.498766 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.498749    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal"
Jul 24 03:17:12.505765 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.505739    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 03:17:12.523865 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.523839    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 03:17:12.524070 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.524039    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 03:17:12.524111 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.524056    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 03:17:12.560314 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.560276    2577 apiserver.go:52] "Watching apiserver"
Jul 24 03:17:12.566108 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.566085    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 03:17:12.566501 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.566475    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-wgmkm","openshift-multus/network-metrics-daemon-8cs8x","openshift-network-diagnostics/network-check-target-qsw5d","openshift-network-operator/iptables-alerter-77wt6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw","openshift-dns/node-resolver-8b2k6","openshift-image-registry/node-ca-lxx2x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal","openshift-multus/multus-vn9dz","openshift-ovn-kubernetes/ovnkube-node-gzp5b","kube-system/global-pull-secret-syncer-4mflr","kube-system/konnectivity-agent-wp6hh","kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal","openshift-cluster-node-tuning-operator/tuned-bgqb9"]
Jul 24 03:17:12.569217 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.569198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.571476 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.571453    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:12.571575 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.571525    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 03:17:12.571712 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.571689    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 03:17:12.571712 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.571693    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 03:17:12.571888 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.571864    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 03:17:12.571940 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.571891    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 03:17:12.572008 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.571993    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 03:17:12.572085 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.572073    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-96hlp\""
Jul 24 03:17:12.573491 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.573476    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 03:17:12.573650 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.573636    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:12.573715 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:12.573695    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:12.573799 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.573785    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 03:17:12.574064 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.574050    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 03:17:12.574117 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.574089    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 03:17:12.574117 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.574048    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xnn7b\""
Jul 24 03:17:12.574296 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.574283    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 03:17:12.576477 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.576459    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:12.578631 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.578610    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 03:17:12.578767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.578650    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:17:12.578767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.578670    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2smvh\""
Jul 24 03:17:12.578767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.578680    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 03:17:12.579097 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.579080    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:12.582319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.581773    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 03:17:12.582319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.581874    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 03:17:12.582319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.582153    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 03:17:12.582319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.582217    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 03:17:12.582567 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.582556    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.582862 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.582843    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ctsbq\""
Jul 24 03:17:12.585112 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.585093    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.587474 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:12.587474 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-ovn-node-metrics-cert\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.587640 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-os-release\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:12.587640 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee30a63f-de99-470a-9f1e-aaba27f385c9-tmp\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.587640 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-run-netns\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.587640 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-hostroot\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.587640 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-cni-netd\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.587640 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-registration-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-kubernetes\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-run\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-var-lib-kubelet\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587733    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-sysctl-conf\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-env-overrides\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-var-lib-openvswitch\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587819    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587829    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-cni-bin\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-var-lib-kubelet\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.587910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-conf-dir\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-etc-kubernetes\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-run-netns\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-log-socket\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.587988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-device-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588025    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-258hn\" (UniqueName: \"kubernetes.io/projected/f8f4d5bb-b5fb-40db-9143-6244c088bea1-kube-api-access-258hn\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b5d8fe8-460b-4a4e-a442-b5d90f290111-cni-binary-copy\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fdf1e4b-579b-4bae-a009-b4bf924bec97-host-slash\") pod \"iptables-alerter-77wt6\" (UID: \"4fdf1e4b-579b-4bae-a009-b4bf924bec97\") " pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588112    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkll5\" (UniqueName: \"kubernetes.io/projected/ee30a63f-de99-470a-9f1e-aaba27f385c9-kube-api-access-rkll5\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-system-cni-dir\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-tuned\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-run-systemd\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-sys\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m7s9\" (UniqueName: \"kubernetes.io/projected/3b5d8fe8-460b-4a4e-a442-b5d90f290111-kube-api-access-9m7s9\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fdf1e4b-579b-4bae-a009-b4bf924bec97-iptables-alerter-script\") pod \"iptables-alerter-77wt6\" (UID: \"4fdf1e4b-579b-4bae-a009-b4bf924bec97\") " pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588275    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-49r5p\""
Jul 24 03:17:12.588429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588290    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-ovnkube-script-lib\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588327    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7jrv2\""
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-sys-fs\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-sysconfig\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-cnibin\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588470    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-socket-dir-parent\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-slash\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-node-log\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-socket-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-host\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-daemon-config\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-run-openvswitch\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-lib-modules\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5bn8\" (UniqueName: \"kubernetes.io/projected/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-kube-api-access-r5bn8\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-kubelet\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.588950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-run-ovn-kubernetes\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-cni-dir\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-modprobe-d\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-ovnkube-config\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv42t\" (UniqueName: \"kubernetes.io/projected/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-kube-api-access-hv42t\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-run-k8s-cni-cncf-io\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588916    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-os-release\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.588979    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-var-lib-cni-multus\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-run-multus-certs\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-cnibin\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-cni-binary-copy\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-sysctl-d\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.589458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-var-lib-cni-bin\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-systemd-units\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-etc-openvswitch\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-run-ovn\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-systemd\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncfdm\" (UniqueName: \"kubernetes.io/projected/4fdf1e4b-579b-4bae-a009-b4bf924bec97-kube-api-access-ncfdm\") pod \"iptables-alerter-77wt6\" (UID: \"4fdf1e4b-579b-4bae-a009-b4bf924bec97\") " pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-system-cni-dir\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589552    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589557    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.589949    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nsnr7\""
Jul 24 03:17:12.590005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:12.590001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:12.590359 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:12.590051    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:14.541170 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.541152    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:14.541290 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:14.541219    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:14.543498 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.543480    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.545763 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.545743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.546245 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.546226    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 03:17:14.546514 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.546247    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 03:17:14.546514 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.546234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-964lf\""
Jul 24 03:17:14.546514 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.546234    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 03:17:14.547556 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.547540    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 03:17:14.547901 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.547884    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 03:17:14.548165 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.548152    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6kzdt\""
Jul 24 03:17:14.551372 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.551353    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 03:17:14.562857 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.562802    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 03:12:13 +0000 UTC" deadline="2028-03-26 19:14:59.715062819 +0000 UTC"
Jul 24 03:17:14.562857 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.562855    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14679h57m45.152210655s"
Jul 24 03:17:14.572060 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.572042    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z6gf6"
Jul 24 03:17:14.579225 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.579203    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z6gf6"
Jul 24 03:17:14.631860 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.631835    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 03:17:14.638301 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638278    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d03b8479-3f64-4961-8243-96e9d6f6d551-host\") pod \"node-ca-lxx2x\" (UID: \"d03b8479-3f64-4961-8243-96e9d6f6d551\") " pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.638416 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-tuned\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.638416 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-run-systemd\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.638416 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-sys\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.638416 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m7s9\" (UniqueName: \"kubernetes.io/projected/3b5d8fe8-460b-4a4e-a442-b5d90f290111-kube-api-access-9m7s9\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.638416 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fdf1e4b-579b-4bae-a009-b4bf924bec97-iptables-alerter-script\") pod \"iptables-alerter-77wt6\" (UID: \"4fdf1e4b-579b-4bae-a009-b4bf924bec97\") " pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-sys\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638438    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-run-systemd\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-ovnkube-script-lib\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-sys-fs\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-sysconfig\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-cnibin\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-socket-dir-parent\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-slash\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-node-log\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638649    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-sysconfig\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.638662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638700    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-socket-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-host\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-cnibin\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638735    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-daemon-config\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-socket-dir-parent\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69bd88a4-0811-4ba8-a935-20746395f8a6-dbus\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638851    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d03b8479-3f64-4961-8243-96e9d6f6d551-serviceca\") pod \"node-ca-lxx2x\" (UID: \"d03b8479-3f64-4961-8243-96e9d6f6d551\") " pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-run-openvswitch\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-lib-modules\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5bn8\" (UniqueName: \"kubernetes.io/projected/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-kube-api-access-r5bn8\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-kubelet\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.638977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-run-ovn-kubernetes\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-cni-dir\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-modprobe-d\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f7984107-7911-40e1-952d-fe9561244ed4-konnectivity-ca\") pod \"konnectivity-agent-wp6hh\" (UID: \"f7984107-7911-40e1-952d-fe9561244ed4\") " pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:14.639174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-ovnkube-config\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639114    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-ovnkube-script-lib\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639155    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv42t\" (UniqueName: \"kubernetes.io/projected/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-kube-api-access-hv42t\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4fdf1e4b-579b-4bae-a009-b4bf924bec97-iptables-alerter-script\") pod \"iptables-alerter-77wt6\" (UID: \"4fdf1e4b-579b-4bae-a009-b4bf924bec97\") " pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-run-k8s-cni-cncf-io\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2ppb\" (UniqueName: \"kubernetes.io/projected/d03b8479-3f64-4961-8243-96e9d6f6d551-kube-api-access-m2ppb\") pod \"node-ca-lxx2x\" (UID: \"d03b8479-3f64-4961-8243-96e9d6f6d551\") " pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-sys-fs\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-run-ovn-kubernetes\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639286    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-slash\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639310    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-os-release\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-socket-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-var-lib-cni-multus\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-node-log\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639354    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-lib-modules\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.639969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-run-multus-certs\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-host\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-cnibin\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-kubelet\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-cni-binary-copy\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-sysctl-d\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-var-lib-cni-bin\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/49dc3ed5-5da8-4347-b531-d650944e4ee8-hosts-file\") pod \"node-resolver-8b2k6\" (UID: \"49dc3ed5-5da8-4347-b531-d650944e4ee8\") " pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-systemd-units\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-etc-openvswitch\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-run-ovn\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-cni-dir\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-systemd\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncfdm\" (UniqueName: \"kubernetes.io/projected/4fdf1e4b-579b-4bae-a009-b4bf924bec97-kube-api-access-ncfdm\") pod \"iptables-alerter-77wt6\" (UID: \"4fdf1e4b-579b-4bae-a009-b4bf924bec97\") " pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-system-cni-dir\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69bd88a4-0811-4ba8-a935-20746395f8a6-kubelet-config\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:14.640669 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639758    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49dc3ed5-5da8-4347-b531-d650944e4ee8-tmp-dir\") pod \"node-resolver-8b2k6\" (UID: \"49dc3ed5-5da8-4347-b531-d650944e4ee8\") " pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-ovn-node-metrics-cert\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639221    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-daemon-config\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-modprobe-d\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-os-release\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee30a63f-de99-470a-9f1e-aaba27f385c9-tmp\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-run-netns\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-hostroot\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqwct\" (UniqueName: \"kubernetes.io/projected/622bcd14-324b-4bae-b7e1-37f16f28c417-kube-api-access-hqwct\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-cni-netd\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-registration-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-kubernetes\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-run\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640131    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-var-lib-kubelet\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f7984107-7911-40e1-952d-fe9561244ed4-agent-certs\") pod \"konnectivity-agent-wp6hh\" (UID: \"f7984107-7911-40e1-952d-fe9561244ed4\") " pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:14.641560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-sysctl-conf\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-env-overrides\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640207    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-var-lib-openvswitch\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-cni-binary-copy\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.639421    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-run-k8s-cni-cncf-io\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640285    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-run-openvswitch\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-cni-bin\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-run\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-var-lib-kubelet\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-var-lib-kubelet\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640403    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-conf-dir\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-run-netns\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-etc-kubernetes\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-run-netns\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640451    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-hostroot\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.642499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-log-socket\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-log-socket\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640507    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-registration-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-device-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640526    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-etc-openvswitch\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-systemd-units\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-258hn\" (UniqueName: \"kubernetes.io/projected/f8f4d5bb-b5fb-40db-9143-6244c088bea1-kube-api-access-258hn\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640565    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-var-lib-openvswitch\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b5d8fe8-460b-4a4e-a442-b5d90f290111-cni-binary-copy\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fdf1e4b-579b-4bae-a009-b4bf924bec97-host-slash\") pod \"iptables-alerter-77wt6\" (UID: \"4fdf1e4b-579b-4bae-a009-b4bf924bec97\") " pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640621    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hft9\" (UniqueName: \"kubernetes.io/projected/49dc3ed5-5da8-4347-b531-d650944e4ee8-kube-api-access-8hft9\") pod \"node-resolver-8b2k6\" (UID: \"49dc3ed5-5da8-4347-b531-d650944e4ee8\") " pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkll5\" (UniqueName: \"kubernetes.io/projected/ee30a63f-de99-470a-9f1e-aaba27f385c9-kube-api-access-rkll5\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640657    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-system-cni-dir\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640723    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-system-cni-dir\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640812    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-sysctl-d\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.643320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640835    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-cni-netd\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640859    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-var-lib-cni-bin\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-os-release\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640919    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.640946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-device-dir\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-env-overrides\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-ovnkube-config\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641129    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-kubernetes\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-sysctl-conf\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-cni-bin\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641340    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641381    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-var-lib-kubelet\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-multus-conf-dir\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b5d8fe8-460b-4a4e-a442-b5d90f290111-cni-binary-copy\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-etc-kubernetes\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-host-run-netns\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fdf1e4b-579b-4bae-a009-b4bf924bec97-host-slash\") pod \"iptables-alerter-77wt6\" (UID: \"4fdf1e4b-579b-4bae-a009-b4bf924bec97\") " pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-cnibin\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.644129 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641588    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-systemd\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.644931 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-var-lib-cni-multus\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.644931 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-os-release\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.644931 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3b5d8fe8-460b-4a4e-a442-b5d90f290111-host-run-multus-certs\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.644931 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641821    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-system-cni-dir\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.644931 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641853    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-run-ovn\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.644931 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.641924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8f4d5bb-b5fb-40db-9143-6244c088bea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.645504 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.645478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee30a63f-de99-470a-9f1e-aaba27f385c9-tmp\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.645602 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.645490    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee30a63f-de99-470a-9f1e-aaba27f385c9-etc-tuned\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.645602 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.645584    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-ovn-node-metrics-cert\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.652944 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:14.652906    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:14.653084 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:14.652950    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:14.653084 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:14.652967    2577 projected.go:196] Error preparing data for projected volume kube-api-access-kbwdt for pod openshift-network-diagnostics/network-check-target-qsw5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:14.653084 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:14.653047    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt podName:ac48200c-f150-494e-9f56-d1fdfc8df128 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:15.153024492 +0000 UTC m=+1.913932952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kbwdt" (UniqueName: "kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt") pod "network-check-target-qsw5d" (UID: "ac48200c-f150-494e-9f56-d1fdfc8df128") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:14.653805 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.653780    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m7s9\" (UniqueName: \"kubernetes.io/projected/3b5d8fe8-460b-4a4e-a442-b5d90f290111-kube-api-access-9m7s9\") pod \"multus-vn9dz\" (UID: \"3b5d8fe8-460b-4a4e-a442-b5d90f290111\") " pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.654428 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.654411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv42t\" (UniqueName: \"kubernetes.io/projected/6c20bb78-3f2a-4df1-ab85-1f38ca5aa566-kube-api-access-hv42t\") pod \"multus-additional-cni-plugins-wgmkm\" (UID: \"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566\") " pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.654928 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.654904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-258hn\" (UniqueName: \"kubernetes.io/projected/f8f4d5bb-b5fb-40db-9143-6244c088bea1-kube-api-access-258hn\") pod \"aws-ebs-csi-driver-node-gkbgw\" (UID: \"f8f4d5bb-b5fb-40db-9143-6244c088bea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.655065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.655039    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkll5\" (UniqueName: \"kubernetes.io/projected/ee30a63f-de99-470a-9f1e-aaba27f385c9-kube-api-access-rkll5\") pod \"tuned-bgqb9\" (UID: \"ee30a63f-de99-470a-9f1e-aaba27f385c9\") " pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.655339 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.655322    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5bn8\" (UniqueName: \"kubernetes.io/projected/cba9a317-933a-4cf0-aee6-b10fa5e6b89c-kube-api-access-r5bn8\") pod \"ovnkube-node-gzp5b\" (UID: \"cba9a317-933a-4cf0-aee6-b10fa5e6b89c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.655410 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.655355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncfdm\" (UniqueName: \"kubernetes.io/projected/4fdf1e4b-579b-4bae-a009-b4bf924bec97-kube-api-access-ncfdm\") pod \"iptables-alerter-77wt6\" (UID: \"4fdf1e4b-579b-4bae-a009-b4bf924bec97\") " pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:14.694052 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.694026    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:14.731409 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.731379    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda261e7af2cd24409d94213307b3a2dcd.slice/crio-88aeaac63923bf45ae5ce6907b3c68b87a32ebc36ffa26d7e4a23a3517c2155d WatchSource:0}: Error finding container 88aeaac63923bf45ae5ce6907b3c68b87a32ebc36ffa26d7e4a23a3517c2155d: Status 404 returned error can't find the container with id 88aeaac63923bf45ae5ce6907b3c68b87a32ebc36ffa26d7e4a23a3517c2155d
Jul 24 03:17:14.731843 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.731816    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4fbcf9cbb15b17f76420beb7ed4f29.slice/crio-3047160e53dcfd468b1a00a08e2389d6bc3b580fcd63ca5fe75e402d706d587e WatchSource:0}: Error finding container 3047160e53dcfd468b1a00a08e2389d6bc3b580fcd63ca5fe75e402d706d587e: Status 404 returned error can't find the container with id 3047160e53dcfd468b1a00a08e2389d6bc3b580fcd63ca5fe75e402d706d587e
Jul 24 03:17:14.736512 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.736498    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 03:17:14.741460 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/49dc3ed5-5da8-4347-b531-d650944e4ee8-hosts-file\") pod \"node-resolver-8b2k6\" (UID: \"49dc3ed5-5da8-4347-b531-d650944e4ee8\") " pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.741582 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:14.741582 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69bd88a4-0811-4ba8-a935-20746395f8a6-kubelet-config\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:14.741582 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49dc3ed5-5da8-4347-b531-d650944e4ee8-tmp-dir\") pod \"node-resolver-8b2k6\" (UID: \"49dc3ed5-5da8-4347-b531-d650944e4ee8\") " pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.741582 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwct\" (UniqueName: \"kubernetes.io/projected/622bcd14-324b-4bae-b7e1-37f16f28c417-kube-api-access-hqwct\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:14.741582 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741537    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/49dc3ed5-5da8-4347-b531-d650944e4ee8-hosts-file\") pod \"node-resolver-8b2k6\" (UID: \"49dc3ed5-5da8-4347-b531-d650944e4ee8\") " pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.741582 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f7984107-7911-40e1-952d-fe9561244ed4-agent-certs\") pod \"konnectivity-agent-wp6hh\" (UID: \"f7984107-7911-40e1-952d-fe9561244ed4\") " pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:14.741582 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69bd88a4-0811-4ba8-a935-20746395f8a6-kubelet-config\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hft9\" (UniqueName: \"kubernetes.io/projected/49dc3ed5-5da8-4347-b531-d650944e4ee8-kube-api-access-8hft9\") pod \"node-resolver-8b2k6\" (UID: \"49dc3ed5-5da8-4347-b531-d650944e4ee8\") " pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d03b8479-3f64-4961-8243-96e9d6f6d551-host\") pod \"node-ca-lxx2x\" (UID: \"d03b8479-3f64-4961-8243-96e9d6f6d551\") " pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:14.741683    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69bd88a4-0811-4ba8-a935-20746395f8a6-dbus\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741732    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d03b8479-3f64-4961-8243-96e9d6f6d551-serviceca\") pod \"node-ca-lxx2x\" (UID: \"d03b8479-3f64-4961-8243-96e9d6f6d551\") " pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:14.741761    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:15.241739712 +0000 UTC m=+2.002648193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f7984107-7911-40e1-952d-fe9561244ed4-konnectivity-ca\") pod \"konnectivity-agent-wp6hh\" (UID: \"f7984107-7911-40e1-952d-fe9561244ed4\") " pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:14.741830    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741836    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49dc3ed5-5da8-4347-b531-d650944e4ee8-tmp-dir\") pod \"node-resolver-8b2k6\" (UID: \"49dc3ed5-5da8-4347-b531-d650944e4ee8\") " pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2ppb\" (UniqueName: \"kubernetes.io/projected/d03b8479-3f64-4961-8243-96e9d6f6d551-kube-api-access-m2ppb\") pod \"node-ca-lxx2x\" (UID: \"d03b8479-3f64-4961-8243-96e9d6f6d551\") " pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:14.741881    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret podName:69bd88a4-0811-4ba8-a935-20746395f8a6 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:15.241866192 +0000 UTC m=+2.002774681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret") pod "global-pull-secret-syncer-4mflr" (UID: "69bd88a4-0811-4ba8-a935-20746395f8a6") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:14.741909 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741905    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d03b8479-3f64-4961-8243-96e9d6f6d551-host\") pod \"node-ca-lxx2x\" (UID: \"d03b8479-3f64-4961-8243-96e9d6f6d551\") " pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.742579 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.741911    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69bd88a4-0811-4ba8-a935-20746395f8a6-dbus\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:14.742579 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.742141    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d03b8479-3f64-4961-8243-96e9d6f6d551-serviceca\") pod \"node-ca-lxx2x\" (UID: \"d03b8479-3f64-4961-8243-96e9d6f6d551\") " pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.742579 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.742394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f7984107-7911-40e1-952d-fe9561244ed4-konnectivity-ca\") pod \"konnectivity-agent-wp6hh\" (UID: \"f7984107-7911-40e1-952d-fe9561244ed4\") " pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:14.744230 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.744214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f7984107-7911-40e1-952d-fe9561244ed4-agent-certs\") pod \"konnectivity-agent-wp6hh\" (UID: \"f7984107-7911-40e1-952d-fe9561244ed4\") " pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:14.749289 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.749246    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hft9\" (UniqueName: \"kubernetes.io/projected/49dc3ed5-5da8-4347-b531-d650944e4ee8-kube-api-access-8hft9\") pod \"node-resolver-8b2k6\" (UID: \"49dc3ed5-5da8-4347-b531-d650944e4ee8\") " pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.749511 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.749493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2ppb\" (UniqueName: \"kubernetes.io/projected/d03b8479-3f64-4961-8243-96e9d6f6d551-kube-api-access-m2ppb\") pod \"node-ca-lxx2x\" (UID: \"d03b8479-3f64-4961-8243-96e9d6f6d551\") " pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.749569 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.749542    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqwct\" (UniqueName: \"kubernetes.io/projected/622bcd14-324b-4bae-b7e1-37f16f28c417-kube-api-access-hqwct\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:14.838182 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.838137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:14.848212 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.848184    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcba9a317_933a_4cf0_aee6_b10fa5e6b89c.slice/crio-9cc518d9f0196b87430e5808881c2d4f05eb4880a955348adf601d4c95db1444 WatchSource:0}: Error finding container 9cc518d9f0196b87430e5808881c2d4f05eb4880a955348adf601d4c95db1444: Status 404 returned error can't find the container with id 9cc518d9f0196b87430e5808881c2d4f05eb4880a955348adf601d4c95db1444
Jul 24 03:17:14.858442 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.858420    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wgmkm"
Jul 24 03:17:14.867372 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.867350    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c20bb78_3f2a_4df1_ab85_1f38ca5aa566.slice/crio-9889665e4de23106f073fc45651b7235cd12c6002f0a8623644f562b2e4e17b5 WatchSource:0}: Error finding container 9889665e4de23106f073fc45651b7235cd12c6002f0a8623644f562b2e4e17b5: Status 404 returned error can't find the container with id 9889665e4de23106f073fc45651b7235cd12c6002f0a8623644f562b2e4e17b5
Jul 24 03:17:14.876617 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.876596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-77wt6"
Jul 24 03:17:14.886625 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.886604    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fdf1e4b_579b_4bae_a009_b4bf924bec97.slice/crio-64caa7ab19470b220610b5697dfeb8f6cb45a62f4351c96908571ec08d821036 WatchSource:0}: Error finding container 64caa7ab19470b220610b5697dfeb8f6cb45a62f4351c96908571ec08d821036: Status 404 returned error can't find the container with id 64caa7ab19470b220610b5697dfeb8f6cb45a62f4351c96908571ec08d821036
Jul 24 03:17:14.889668 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.889640    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw"
Jul 24 03:17:14.894659 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.894638    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bgqb9"
Jul 24 03:17:14.898362 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.898314    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f4d5bb_b5fb_40db_9143_6244c088bea1.slice/crio-1349c506edfaf014e753897862bbb26207f0d0df1e56a978cd6fd60b4fbe9335 WatchSource:0}: Error finding container 1349c506edfaf014e753897862bbb26207f0d0df1e56a978cd6fd60b4fbe9335: Status 404 returned error can't find the container with id 1349c506edfaf014e753897862bbb26207f0d0df1e56a978cd6fd60b4fbe9335
Jul 24 03:17:14.903339 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.903315    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee30a63f_de99_470a_9f1e_aaba27f385c9.slice/crio-b5460d0a954e986850d52681acdaaae65e6ccd1e1e6f012b7dfb61a960af6593 WatchSource:0}: Error finding container b5460d0a954e986850d52681acdaaae65e6ccd1e1e6f012b7dfb61a960af6593: Status 404 returned error can't find the container with id b5460d0a954e986850d52681acdaaae65e6ccd1e1e6f012b7dfb61a960af6593
Jul 24 03:17:14.915830 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.915800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vn9dz"
Jul 24 03:17:14.925458 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.925426    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5d8fe8_460b_4a4e_a442_b5d90f290111.slice/crio-b543adedaccf64ef5c80e59f417ec14980b8575e3c48b21d14f094d4454a4af1 WatchSource:0}: Error finding container b543adedaccf64ef5c80e59f417ec14980b8575e3c48b21d14f094d4454a4af1: Status 404 returned error can't find the container with id b543adedaccf64ef5c80e59f417ec14980b8575e3c48b21d14f094d4454a4af1
Jul 24 03:17:14.929606 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.929587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:14.935583 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.935566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lxx2x"
Jul 24 03:17:14.938383 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.938203    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7984107_7911_40e1_952d_fe9561244ed4.slice/crio-f971e0db5a7b4af36aa45a2a01dfa8a84b9c80275c940bb84ad2742f5dfe8394 WatchSource:0}: Error finding container f971e0db5a7b4af36aa45a2a01dfa8a84b9c80275c940bb84ad2742f5dfe8394: Status 404 returned error can't find the container with id f971e0db5a7b4af36aa45a2a01dfa8a84b9c80275c940bb84ad2742f5dfe8394
Jul 24 03:17:14.944293 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.944256    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd03b8479_3f64_4961_8243_96e9d6f6d551.slice/crio-8d19cdfaeb02b8a1b00001a5f89e37b7ce0377b01a4709d968be927a166f0ed2 WatchSource:0}: Error finding container 8d19cdfaeb02b8a1b00001a5f89e37b7ce0377b01a4709d968be927a166f0ed2: Status 404 returned error can't find the container with id 8d19cdfaeb02b8a1b00001a5f89e37b7ce0377b01a4709d968be927a166f0ed2
Jul 24 03:17:14.945762 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:14.945746    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8b2k6"
Jul 24 03:17:14.954793 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:14.954771    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49dc3ed5_5da8_4347_b531_d650944e4ee8.slice/crio-9f35bb28f4ec95707fc58460394c6605b28494780acba44bef30072fbf934bff WatchSource:0}: Error finding container 9f35bb28f4ec95707fc58460394c6605b28494780acba44bef30072fbf934bff: Status 404 returned error can't find the container with id 9f35bb28f4ec95707fc58460394c6605b28494780acba44bef30072fbf934bff
Jul 24 03:17:15.245908 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.245865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:15.246090 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.245935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:15.246090 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.245971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:15.246090 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:15.246029    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:15.246090 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:15.246077    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:15.246090 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:15.246089    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:16.246074972 +0000 UTC m=+3.006983430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:15.246390 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:15.246128    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret podName:69bd88a4-0811-4ba8-a935-20746395f8a6 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:16.246112564 +0000 UTC m=+3.007021027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret") pod "global-pull-secret-syncer-4mflr" (UID: "69bd88a4-0811-4ba8-a935-20746395f8a6") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:15.246390 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:15.246031    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:15.246390 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:15.246177    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:15.246390 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:15.246195    2577 projected.go:196] Error preparing data for projected volume kube-api-access-kbwdt for pod openshift-network-diagnostics/network-check-target-qsw5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:15.246390 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:15.246251    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt podName:ac48200c-f150-494e-9f56-d1fdfc8df128 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:16.246235021 +0000 UTC m=+3.007143499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kbwdt" (UniqueName: "kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt") pod "network-check-target-qsw5d" (UID: "ac48200c-f150-494e-9f56-d1fdfc8df128") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:15.584138 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.583233    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 03:12:14 +0000 UTC" deadline="2028-03-24 02:24:11.212640068 +0000 UTC"
Jul 24 03:17:15.584138 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.583310    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14615h6m55.629334257s"
Jul 24 03:17:15.647449 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.647329    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lxx2x" event={"ID":"d03b8479-3f64-4961-8243-96e9d6f6d551","Type":"ContainerStarted","Data":"8d19cdfaeb02b8a1b00001a5f89e37b7ce0377b01a4709d968be927a166f0ed2"}
Jul 24 03:17:15.650029 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.649943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bgqb9" event={"ID":"ee30a63f-de99-470a-9f1e-aaba27f385c9","Type":"ContainerStarted","Data":"b5460d0a954e986850d52681acdaaae65e6ccd1e1e6f012b7dfb61a960af6593"}
Jul 24 03:17:15.652752 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.652626    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw" event={"ID":"f8f4d5bb-b5fb-40db-9143-6244c088bea1","Type":"ContainerStarted","Data":"1349c506edfaf014e753897862bbb26207f0d0df1e56a978cd6fd60b4fbe9335"}
Jul 24 03:17:15.655708 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.655642    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-77wt6" event={"ID":"4fdf1e4b-579b-4bae-a009-b4bf924bec97","Type":"ContainerStarted","Data":"64caa7ab19470b220610b5697dfeb8f6cb45a62f4351c96908571ec08d821036"}
Jul 24 03:17:15.660360 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.660139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgmkm" event={"ID":"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566","Type":"ContainerStarted","Data":"9889665e4de23106f073fc45651b7235cd12c6002f0a8623644f562b2e4e17b5"}
Jul 24 03:17:15.665694 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.665620    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal" event={"ID":"a261e7af2cd24409d94213307b3a2dcd","Type":"ContainerStarted","Data":"88aeaac63923bf45ae5ce6907b3c68b87a32ebc36ffa26d7e4a23a3517c2155d"}
Jul 24 03:17:15.675099 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.675070    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal" event={"ID":"3a4fbcf9cbb15b17f76420beb7ed4f29","Type":"ContainerStarted","Data":"3047160e53dcfd468b1a00a08e2389d6bc3b580fcd63ca5fe75e402d706d587e"}
Jul 24 03:17:15.677380 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.677316    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8b2k6" event={"ID":"49dc3ed5-5da8-4347-b531-d650944e4ee8","Type":"ContainerStarted","Data":"9f35bb28f4ec95707fc58460394c6605b28494780acba44bef30072fbf934bff"}
Jul 24 03:17:15.683228 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.683161    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wp6hh" event={"ID":"f7984107-7911-40e1-952d-fe9561244ed4","Type":"ContainerStarted","Data":"f971e0db5a7b4af36aa45a2a01dfa8a84b9c80275c940bb84ad2742f5dfe8394"}
Jul 24 03:17:15.689371 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.689322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vn9dz" event={"ID":"3b5d8fe8-460b-4a4e-a442-b5d90f290111","Type":"ContainerStarted","Data":"b543adedaccf64ef5c80e59f417ec14980b8575e3c48b21d14f094d4454a4af1"}
Jul 24 03:17:15.694740 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.694681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" event={"ID":"cba9a317-933a-4cf0-aee6-b10fa5e6b89c","Type":"ContainerStarted","Data":"9cc518d9f0196b87430e5808881c2d4f05eb4880a955348adf601d4c95db1444"}
Jul 24 03:17:15.913923 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:15.913890    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:16.161049 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:16.161012    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:16.257123 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:16.257044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:16.257295 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:16.257120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:16.257295 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:16.257162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:16.257420 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.257315    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:16.257420 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.257383    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:18.257362745 +0000 UTC m=+5.018271217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:16.257420 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.257398    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:16.257568 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.257461    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret podName:69bd88a4-0811-4ba8-a935-20746395f8a6 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:18.257444608 +0000 UTC m=+5.018353069 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret") pod "global-pull-secret-syncer-4mflr" (UID: "69bd88a4-0811-4ba8-a935-20746395f8a6") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:16.257568 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.257555    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:16.257675 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.257572    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:16.257675 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.257585    2577 projected.go:196] Error preparing data for projected volume kube-api-access-kbwdt for pod openshift-network-diagnostics/network-check-target-qsw5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:16.257675 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.257621    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt podName:ac48200c-f150-494e-9f56-d1fdfc8df128 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:18.257609305 +0000 UTC m=+5.018517769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kbwdt" (UniqueName: "kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt") pod "network-check-target-qsw5d" (UID: "ac48200c-f150-494e-9f56-d1fdfc8df128") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:16.584537 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:16.584443    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 03:12:14 +0000 UTC" deadline="2028-03-25 03:01:56.18886095 +0000 UTC"
Jul 24 03:17:16.584537 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:16.584486    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14639h44m39.604379787s"
Jul 24 03:17:16.637782 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:16.636959    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:16.637782 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.637090    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:16.637782 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:16.637496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:16.637782 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.637584    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:16.637782 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:16.637647    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:16.637782 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:16.637729    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:18.276232 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:18.276195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:18.276259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:18.276309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.276398    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.276427    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.276438    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.276441    2577 projected.go:196] Error preparing data for projected volume kube-api-access-kbwdt for pod openshift-network-diagnostics/network-check-target-qsw5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.276496    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret podName:69bd88a4-0811-4ba8-a935-20746395f8a6 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:22.276474932 +0000 UTC m=+9.037383409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret") pod "global-pull-secret-syncer-4mflr" (UID: "69bd88a4-0811-4ba8-a935-20746395f8a6") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.276514    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt podName:ac48200c-f150-494e-9f56-d1fdfc8df128 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:22.276505471 +0000 UTC m=+9.037413929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kbwdt" (UniqueName: "kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt") pod "network-check-target-qsw5d" (UID: "ac48200c-f150-494e-9f56-d1fdfc8df128") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.276541    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:18.276715 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.276583    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:22.276565902 +0000 UTC m=+9.037474362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:18.637182 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:18.637096    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:18.637380 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.637229    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:18.637671 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:18.637651    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:18.637786 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.637760    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:18.637842 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:18.637824    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:18.637914 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:18.637896    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:20.636646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:20.636607    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:20.636646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:20.636643    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:20.637185 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:20.636650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:20.637185 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:20.636757    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:20.637185 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:20.636824    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:20.637185 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:20.636912    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:22.308363 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:22.308330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:22.308363 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:22.308380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:22.308893 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:22.308414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:22.308893 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.308549    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:22.308893 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.308550    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:22.308893 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.308575    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:22.308893 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.308589    2577 projected.go:196] Error preparing data for projected volume kube-api-access-kbwdt for pod openshift-network-diagnostics/network-check-target-qsw5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:22.308893 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.308599    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret podName:69bd88a4-0811-4ba8-a935-20746395f8a6 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:30.30858509 +0000 UTC m=+17.069493548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret") pod "global-pull-secret-syncer-4mflr" (UID: "69bd88a4-0811-4ba8-a935-20746395f8a6") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:22.308893 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.308645    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt podName:ac48200c-f150-494e-9f56-d1fdfc8df128 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:30.308626447 +0000 UTC m=+17.069534909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kbwdt" (UniqueName: "kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt") pod "network-check-target-qsw5d" (UID: "ac48200c-f150-494e-9f56-d1fdfc8df128") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:22.308893 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.308709    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:22.308893 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.308741    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:30.308730596 +0000 UTC m=+17.069639054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:22.636677 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:22.636603    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:22.636841 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:22.636746    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:22.636841 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.636760    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:22.636957 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.636874    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:22.637660 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:22.637347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:22.638003 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:22.637961    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:24.637050 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:24.637016    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:24.637530 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:24.637069    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:24.637530 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:24.637207    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:24.637530 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:24.637317    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:24.637530 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:24.637348    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:24.637530 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:24.637450    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:26.636970 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:26.636433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:26.636970 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:26.636475    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:26.636970 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:26.636433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:26.636970 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:26.636585    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:26.636970 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:26.636687    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:26.636970 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:26.636775    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:28.636989 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:28.636951    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:28.637472 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:28.636961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:28.637472 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:28.637119    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:28.637472 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:28.637175    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:28.637472 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:28.637301    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:28.637472 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:28.637395    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:30.371614 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:30.371570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:30.371630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:30.371660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.371789    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.371825    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.371825    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.371837    2577 projected.go:196] Error preparing data for projected volume kube-api-access-kbwdt for pod openshift-network-diagnostics/network-check-target-qsw5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.371787    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.371888    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt podName:ac48200c-f150-494e-9f56-d1fdfc8df128 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:46.371868666 +0000 UTC m=+33.132777139 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kbwdt" (UniqueName: "kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt") pod "network-check-target-qsw5d" (UID: "ac48200c-f150-494e-9f56-d1fdfc8df128") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.371913    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:46.371895067 +0000 UTC m=+33.132803524 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:30.372068 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.371933    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret podName:69bd88a4-0811-4ba8-a935-20746395f8a6 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:46.371922545 +0000 UTC m=+33.132831002 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret") pod "global-pull-secret-syncer-4mflr" (UID: "69bd88a4-0811-4ba8-a935-20746395f8a6") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:30.636752 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:30.636657    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:30.636752 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:30.636689    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:30.636970 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:30.636656    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:30.636970 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.636823    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:30.636970 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.636929    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:30.637115 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:30.637039    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:32.636983 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:32.636452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:32.636983 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:32.636469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:32.636983 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:32.636580    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:32.636983 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:32.636889    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:32.637654 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:32.636997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:32.637654 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:32.637103    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:34.636336 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:34.636306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:34.636769 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:34.636313    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:34.636769 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:34.636450    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:34.636769 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:34.636541    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:34.636769 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:34.636556    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:34.636769 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:34.636642    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:35.792801 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.792500    2577 generic.go:356] "Generic (PLEG): container finished" podID="a261e7af2cd24409d94213307b3a2dcd" containerID="4660d93ea9fd994a5e62188b40d5c5f88843c4a46cbed21e0cea69079b486cd7" exitCode=0
Jul 24 03:17:35.793534 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.792582    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal" event={"ID":"a261e7af2cd24409d94213307b3a2dcd","Type":"ContainerDied","Data":"4660d93ea9fd994a5e62188b40d5c5f88843c4a46cbed21e0cea69079b486cd7"}
Jul 24 03:17:35.794415 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.794388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal" event={"ID":"3a4fbcf9cbb15b17f76420beb7ed4f29","Type":"ContainerStarted","Data":"14fc5e56ccc484ee751419e35cd16dcfec8c5883a635a17357e422dda02b8beb"}
Jul 24 03:17:35.795726 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.795705    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8b2k6" event={"ID":"49dc3ed5-5da8-4347-b531-d650944e4ee8","Type":"ContainerStarted","Data":"205605b0fd7c1021aa161320f99f6d1262b33ea13f680a0e34cfd7f366168eaa"}
Jul 24 03:17:35.800018 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.799983    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wp6hh" event={"ID":"f7984107-7911-40e1-952d-fe9561244ed4","Type":"ContainerStarted","Data":"9c5ab94594bfd23ad4751e744f2d64e3099a542831e17e9c835902c484063e88"}
Jul 24 03:17:35.801425 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.801403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vn9dz" event={"ID":"3b5d8fe8-460b-4a4e-a442-b5d90f290111","Type":"ContainerStarted","Data":"4c17a98614bf10eca68e1604cf17722cbf0877c6dacb7f2c65d9dc2bdd96cbd3"}
Jul 24 03:17:35.804061 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.804037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" event={"ID":"cba9a317-933a-4cf0-aee6-b10fa5e6b89c","Type":"ContainerStarted","Data":"8a624a9d5e907498cecd467407f6867c4e1b96deaf321110af9e76aaa08d2f49"}
Jul 24 03:17:35.804171 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.804108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" event={"ID":"cba9a317-933a-4cf0-aee6-b10fa5e6b89c","Type":"ContainerStarted","Data":"c7b1f30a39cbdcaf1262438a02a9e3e2c2eed523ec6e9572c0370fd70c2befdf"}
Jul 24 03:17:35.804171 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.804147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" event={"ID":"cba9a317-933a-4cf0-aee6-b10fa5e6b89c","Type":"ContainerStarted","Data":"410246c3331f2cf4a31ef3dd53cac09fbdfefcef67a5f7850cb67c962ff14d68"}
Jul 24 03:17:35.804288 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.804182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" event={"ID":"cba9a317-933a-4cf0-aee6-b10fa5e6b89c","Type":"ContainerStarted","Data":"fa0821a069599e927682dfa27c13bc7e8e60f7f4fa0801c0e2e8479dba410d48"}
Jul 24 03:17:35.804288 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.804210    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" event={"ID":"cba9a317-933a-4cf0-aee6-b10fa5e6b89c","Type":"ContainerStarted","Data":"2605b452d81f2353137d9da305439ef1e53c8c5e4e188cd566017f8e312cb91c"}
Jul 24 03:17:35.804288 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.804244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" event={"ID":"cba9a317-933a-4cf0-aee6-b10fa5e6b89c","Type":"ContainerStarted","Data":"8605e8c274b42feebb3c7e6e8ea345b0adfa80e95ceef372fe07a58f861a15ae"}
Jul 24 03:17:35.805293 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.805260    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lxx2x" event={"ID":"d03b8479-3f64-4961-8243-96e9d6f6d551","Type":"ContainerStarted","Data":"108d543b3c79599b4a7555ead5ed417871e6d1994e8cc881f13bc009c0e7c25f"}
Jul 24 03:17:35.806443 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.806425    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bgqb9" event={"ID":"ee30a63f-de99-470a-9f1e-aaba27f385c9","Type":"ContainerStarted","Data":"57d570da2a7ef4c4b5b5afac6798b712228c8b9830f08bfdf3b2e5be692ada84"}
Jul 24 03:17:35.807722 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.807702    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw" event={"ID":"f8f4d5bb-b5fb-40db-9143-6244c088bea1","Type":"ContainerStarted","Data":"16d8625642626dd5e80d4ca40f6ba6c2eaefde3fc2bc97c0989c95360fe49bcb"}
Jul 24 03:17:35.809073 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.809053    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c20bb78-3f2a-4df1-ab85-1f38ca5aa566" containerID="178ee5d2a0e4269fff61918dc91c8dac26fa03c43f08a464cea3b3b9368d20b7" exitCode=0
Jul 24 03:17:35.809166 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.809079    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgmkm" event={"ID":"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566","Type":"ContainerDied","Data":"178ee5d2a0e4269fff61918dc91c8dac26fa03c43f08a464cea3b3b9368d20b7"}
Jul 24 03:17:35.834284 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.834219    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-49.ec2.internal" podStartSLOduration=21.834200147 podStartE2EDuration="21.834200147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:35.833825922 +0000 UTC m=+22.594734403" watchObservedRunningTime="2026-07-24 03:17:35.834200147 +0000 UTC m=+22.595108628"
Jul 24 03:17:35.834678 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.834647    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8b2k6" podStartSLOduration=2.179914311 podStartE2EDuration="21.83463761s" totalImagesPullingTime="19.654723299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:35.821047649 +0000 UTC m=+22.581956128" watchObservedRunningTime="2026-07-24 03:17:35.83463761 +0000 UTC m=+22.595546089"
Jul 24 03:17:35.854314 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.854194    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vn9dz" podStartSLOduration=3.011304257 podStartE2EDuration="22.854174418s" totalImagesPullingTime="19.842870161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:35.853209136 +0000 UTC m=+22.614117618" watchObservedRunningTime="2026-07-24 03:17:35.854174418 +0000 UTC m=+22.615082899"
Jul 24 03:17:35.870750 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.870708    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wp6hh" podStartSLOduration=3.400857877 podStartE2EDuration="22.870693553s" totalImagesPullingTime="19.469835676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:35.870238366 +0000 UTC m=+22.631146847" watchObservedRunningTime="2026-07-24 03:17:35.870693553 +0000 UTC m=+22.631602035"
Jul 24 03:17:35.887227 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.887059    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lxx2x" podStartSLOduration=2.221283352 podStartE2EDuration="21.887045505s" totalImagesPullingTime="19.665762153s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:35.886492321 +0000 UTC m=+22.647400805" watchObservedRunningTime="2026-07-24 03:17:35.887045505 +0000 UTC m=+22.647954019"
Jul 24 03:17:35.902848 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:35.902804    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bgqb9" podStartSLOduration=3.038585415 podStartE2EDuration="22.902789409s" totalImagesPullingTime="19.864203994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:35.902694364 +0000 UTC m=+22.663602844" watchObservedRunningTime="2026-07-24 03:17:35.902789409 +0000 UTC m=+22.663697889"
Jul 24 03:17:36.624613 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:36.624582    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 03:17:36.636951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:36.636869    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:36.636951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:36.636897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:36.637113 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:36.636985    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:36.637113 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:36.637062    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:36.637113 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:36.637108    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:36.637204 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:36.637182    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:36.814336 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:36.814230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw" event={"ID":"f8f4d5bb-b5fb-40db-9143-6244c088bea1","Type":"ContainerStarted","Data":"c05d2be764fe978267c11372b0c306bb49b1f9cd4cf3e1bb998f038bcdcc6ef3"}
Jul 24 03:17:36.816107 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:36.816072    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-77wt6" event={"ID":"4fdf1e4b-579b-4bae-a009-b4bf924bec97","Type":"ContainerStarted","Data":"08603e3800b983f84642ed74a3c3b18bd7f39c9800bdcfde76083507e05e6a0b"}
Jul 24 03:17:36.818223 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:36.818140    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal" event={"ID":"a261e7af2cd24409d94213307b3a2dcd","Type":"ContainerStarted","Data":"843ec75db6b8b9d53895b9df92c6e77dae0148f9e7a00650ea1dcd0502a2b53a"}
Jul 24 03:17:36.834001 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:36.833952    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-77wt6" podStartSLOduration=4.110022188 podStartE2EDuration="23.833934632s" totalImagesPullingTime="19.723912444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:36.833615423 +0000 UTC m=+23.594523902" watchObservedRunningTime="2026-07-24 03:17:36.833934632 +0000 UTC m=+23.594843113"
Jul 24 03:17:36.853817 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:36.853765    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-49.ec2.internal" podStartSLOduration=22.85374818 podStartE2EDuration="22.85374818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:36.853299549 +0000 UTC m=+23.614208025" watchObservedRunningTime="2026-07-24 03:17:36.85374818 +0000 UTC m=+23.614656663"
Jul 24 03:17:37.600499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:37.600392    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T03:17:36.624602048Z","UUID":"0e5a5d03-cde5-4529-99f3-f6917280327c","Handler":null,"Name":"","Endpoint":""}
Jul 24 03:17:37.602175 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:37.602141    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 03:17:37.602329 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:37.602192    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 03:17:38.543707 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:38.543670    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:38.636643 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:38.636554    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:38.636643 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:38.636661    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:38.636643 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:38.636671    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:38.636643 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:38.636693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:38.637307 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:38.636781    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:38.637307 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:38.636863    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:38.826279 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:38.826184    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" event={"ID":"cba9a317-933a-4cf0-aee6-b10fa5e6b89c","Type":"ContainerStarted","Data":"3e193b0627fb003087f74df142e422f06df760d8459dbb5900dbf4fe64a0798a"}
Jul 24 03:17:38.828002 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:38.827976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw" event={"ID":"f8f4d5bb-b5fb-40db-9143-6244c088bea1","Type":"ContainerStarted","Data":"993c6339316208e9be3be51bb89197b12a66ba2658dda74b0b1425167cc5bbd3"}
Jul 24 03:17:38.845491 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:38.845447    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gkbgw" podStartSLOduration=3.168666891 podStartE2EDuration="25.845433252s" totalImagesPullingTime="22.676766361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:38.845092863 +0000 UTC m=+25.606001343" watchObservedRunningTime="2026-07-24 03:17:38.845433252 +0000 UTC m=+25.606341731"
Jul 24 03:17:40.374656 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:40.374618    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:40.375513 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:40.375482    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:40.637084 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:40.636991    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:40.637084 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:40.637015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:40.637084 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:40.637037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:40.637356 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:40.637134    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:40.637356 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:40.637281    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:40.637444 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:40.637390    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:40.833393 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:40.833365    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wp6hh"
Jul 24 03:17:41.838652 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:41.838434    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" event={"ID":"cba9a317-933a-4cf0-aee6-b10fa5e6b89c","Type":"ContainerStarted","Data":"2481d9dcfed7ce88e720196355e918e3c2729a06a012fa331f904ad2db0ffa5b"}
Jul 24 03:17:41.839406 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:41.838705    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:41.839406 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:41.838746    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:41.839406 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:41.838784    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:41.841251 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:41.841227    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c20bb78-3f2a-4df1-ab85-1f38ca5aa566" containerID="5c57943bdaa7f9da658f8e695e1b857d6df042f5e6f814e8d54abf70ad2bade5" exitCode=0
Jul 24 03:17:41.841389 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:41.841257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgmkm" event={"ID":"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566","Type":"ContainerDied","Data":"5c57943bdaa7f9da658f8e695e1b857d6df042f5e6f814e8d54abf70ad2bade5"}
Jul 24 03:17:41.855455 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:41.855428    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:41.855573 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:41.855544    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:17:41.864852 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:41.864805    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b" podStartSLOduration=8.842991889 podStartE2EDuration="28.864789822s" totalImagesPullingTime="20.021797933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:41.863237139 +0000 UTC m=+28.624145625" watchObservedRunningTime="2026-07-24 03:17:41.864789822 +0000 UTC m=+28.625698302"
Jul 24 03:17:42.637043 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:42.637009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:42.637233 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:42.637107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:42.637233 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:42.637115    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:42.637233 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:42.637145    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:42.637387 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:42.637249    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:42.637387 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:42.637342    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:43.253480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:43.253449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4mflr"]
Jul 24 03:17:43.254004 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:43.253577    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:43.254004 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:43.253702    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:43.256459 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:43.256432    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8cs8x"]
Jul 24 03:17:43.256627 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:43.256542    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:43.256688 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:43.256659    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:43.257070 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:43.257049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qsw5d"]
Jul 24 03:17:43.257161 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:43.257127    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:43.257218 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:43.257200    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:43.848844 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:43.848807    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c20bb78-3f2a-4df1-ab85-1f38ca5aa566" containerID="0dd4fbf3d340e335ecd141b351eee2a025cea82fb6b2eef6613308e1c6d7df51" exitCode=0
Jul 24 03:17:43.849064 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:43.848897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgmkm" event={"ID":"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566","Type":"ContainerDied","Data":"0dd4fbf3d340e335ecd141b351eee2a025cea82fb6b2eef6613308e1c6d7df51"}
Jul 24 03:17:44.636969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:44.636728    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:44.637486 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:44.636797    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:44.637486 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:44.637029    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:44.637486 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:44.637058    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:45.636720 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:45.636634    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:45.636894 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:45.636749    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsw5d" podUID="ac48200c-f150-494e-9f56-d1fdfc8df128"
Jul 24 03:17:45.855946 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:45.855907    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c20bb78-3f2a-4df1-ab85-1f38ca5aa566" containerID="12d9e8bb81e77f97515d23512717edfb7856e72cf888b728174307547e21e108" exitCode=0
Jul 24 03:17:45.856341 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:45.855954    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgmkm" event={"ID":"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566","Type":"ContainerDied","Data":"12d9e8bb81e77f97515d23512717edfb7856e72cf888b728174307547e21e108"}
Jul 24 03:17:46.381035 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:46.380993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:46.381035 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:46.381037    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:46.381334 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:46.381071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:46.381334 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.381168    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:46.381334 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.381165    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:46.381334 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.381183    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:46.381334 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.381202    2577 projected.go:196] Error preparing data for projected volume kube-api-access-kbwdt for pod openshift-network-diagnostics/network-check-target-qsw5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:46.381334 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.381166    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:46.381334 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.381235    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret podName:69bd88a4-0811-4ba8-a935-20746395f8a6 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:18.381218273 +0000 UTC m=+65.142126751 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret") pod "global-pull-secret-syncer-4mflr" (UID: "69bd88a4-0811-4ba8-a935-20746395f8a6") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:46.381334 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.381259    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:18.381242902 +0000 UTC m=+65.142151375 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:46.381334 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.381305    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt podName:ac48200c-f150-494e-9f56-d1fdfc8df128 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:18.381294092 +0000 UTC m=+65.142202557 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kbwdt" (UniqueName: "kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt") pod "network-check-target-qsw5d" (UID: "ac48200c-f150-494e-9f56-d1fdfc8df128") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:46.636467 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:46.636386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:46.636618 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.636495    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4mflr" podUID="69bd88a4-0811-4ba8-a935-20746395f8a6"
Jul 24 03:17:46.636618 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:46.636523    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:46.636618 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:46.636580    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:17:47.578488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.578412    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-49.ec2.internal" event="NodeReady"
Jul 24 03:17:47.578837 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.578546    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 03:17:47.613810 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.613770    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"]
Jul 24 03:17:47.631555 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.631511    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d7df49bb-g94h2"]
Jul 24 03:17:47.631728 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.631650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:47.635245 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.635214    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 24 03:17:47.635698 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.635675    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 24 03:17:47.635917 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.635902    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9jmtj\""
Jul 24 03:17:47.649283 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.649227    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"]
Jul 24 03:17:47.650251 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.650227    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.651771 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.651739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:17:47.652861 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.652841    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 03:17:47.653062 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.653045    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 03:17:47.653147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.653124    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bwrkh\""
Jul 24 03:17:47.653212 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.653131    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 03:17:47.654092 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.654067    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-prsrs"]
Jul 24 03:17:47.654249 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.654228    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 03:17:47.655089 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.655072    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 03:17:47.655403 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.655385    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n6kfx\""
Jul 24 03:17:47.659957 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.659938    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 03:17:47.672134 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.672110    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6xp58"]
Jul 24 03:17:47.672391 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.672370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prsrs"
Jul 24 03:17:47.674906 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.674547    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k789z\""
Jul 24 03:17:47.674906 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.674619    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 03:17:47.674906 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.674621    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 03:17:47.674906 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.674773    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 03:17:47.692117 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.692088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:47.692250 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.692156    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efaf2452-d67f-4599-998a-4a496d2c9c07-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:47.695882 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.695862    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d7df49bb-g94h2"]
Jul 24 03:17:47.695952 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.695897    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prsrs"]
Jul 24 03:17:47.695952 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.695916    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6xp58"]
Jul 24 03:17:47.696027 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.695971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.698483 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.698458    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 03:17:47.698483 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.698475    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 03:17:47.698770 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.698489    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fzmf8\""
Jul 24 03:17:47.768645 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.768610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wp6hh_f7984107-7911-40e1-952d-fe9561244ed4/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:47.792498 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efaf2452-d67f-4599-998a-4a496d2c9c07-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:47.792716 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.792716 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-trusted-ca\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.792716 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-certificates\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.792716 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cd4908e-c044-40f4-88b4-6606c01d3054-tmp-dir\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.792960 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lg95\" (UniqueName: \"kubernetes.io/projected/8b304cd9-4445-4be1-921f-0797a250e25d-kube-api-access-7lg95\") pod \"ingress-canary-prsrs\" (UID: \"8b304cd9-4445-4be1-921f-0797a250e25d\") " pod="openshift-ingress-canary/ingress-canary-prsrs"
Jul 24 03:17:47.792960 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792781    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b304cd9-4445-4be1-921f-0797a250e25d-cert\") pod \"ingress-canary-prsrs\" (UID: \"8b304cd9-4445-4be1-921f-0797a250e25d\") " pod="openshift-ingress-canary/ingress-canary-prsrs"
Jul 24 03:17:47.792960 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-image-registry-private-configuration\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.792960 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15f782ab-f003-4bf8-8c63-4343d1afcc61-ca-trust-extracted\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.792960 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.793198 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.792986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:47.793198 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.793020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-installation-pull-secrets\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.793198 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.793047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-bound-sa-token\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.793198 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:47.793074    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 03:17:47.793198 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.793128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mm5b\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-kube-api-access-8mm5b\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.793198 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.793153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6cd4908e-c044-40f4-88b4-6606c01d3054-config-volume\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.793198 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.793175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq69g\" (UniqueName: \"kubernetes.io/projected/6cd4908e-c044-40f4-88b4-6606c01d3054-kube-api-access-nq69g\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.793538 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.793219    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efaf2452-d67f-4599-998a-4a496d2c9c07-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:47.793538 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:47.793288    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert podName:efaf2452-d67f-4599-998a-4a496d2c9c07 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:48.293245354 +0000 UTC m=+35.054153812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sxk8t" (UID: "efaf2452-d67f-4599-998a-4a496d2c9c07") : secret "networking-console-plugin-cert" not found
Jul 24 03:17:47.848190 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.848082    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wp6hh_f7984107-7911-40e1-952d-fe9561244ed4/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:47.893494 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-certificates\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.893494 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cd4908e-c044-40f4-88b4-6606c01d3054-tmp-dir\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.893754 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lg95\" (UniqueName: \"kubernetes.io/projected/8b304cd9-4445-4be1-921f-0797a250e25d-kube-api-access-7lg95\") pod \"ingress-canary-prsrs\" (UID: \"8b304cd9-4445-4be1-921f-0797a250e25d\") " pod="openshift-ingress-canary/ingress-canary-prsrs"
Jul 24 03:17:47.893754 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b304cd9-4445-4be1-921f-0797a250e25d-cert\") pod \"ingress-canary-prsrs\" (UID: \"8b304cd9-4445-4be1-921f-0797a250e25d\") " pod="openshift-ingress-canary/ingress-canary-prsrs"
Jul 24 03:17:47.893754 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-image-registry-private-configuration\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.893754 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15f782ab-f003-4bf8-8c63-4343d1afcc61-ca-trust-extracted\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.893754 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.893754 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-installation-pull-secrets\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.893754 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-bound-sa-token\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.894065 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:47.893753    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:17:47.894065 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:47.893839    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls podName:6cd4908e-c044-40f4-88b4-6606c01d3054 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:48.393815484 +0000 UTC m=+35.154723944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls") pod "dns-default-6xp58" (UID: "6cd4908e-c044-40f4-88b4-6606c01d3054") : secret "dns-default-metrics-tls" not found
Jul 24 03:17:47.894065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mm5b\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-kube-api-access-8mm5b\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.894065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cd4908e-c044-40f4-88b4-6606c01d3054-tmp-dir\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.894065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6cd4908e-c044-40f4-88b4-6606c01d3054-config-volume\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.894065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq69g\" (UniqueName: \"kubernetes.io/projected/6cd4908e-c044-40f4-88b4-6606c01d3054-kube-api-access-nq69g\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.894065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.894065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.893982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-trusted-ca\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.894065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.894053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15f782ab-f003-4bf8-8c63-4343d1afcc61-ca-trust-extracted\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.894522 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.894208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-certificates\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.894522 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:47.894368    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:17:47.894522 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:47.894384    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7df49bb-g94h2: secret "image-registry-tls" not found
Jul 24 03:17:47.894522 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:47.894441    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls podName:15f782ab-f003-4bf8-8c63-4343d1afcc61 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:48.394424654 +0000 UTC m=+35.155333126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls") pod "image-registry-d7df49bb-g94h2" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61") : secret "image-registry-tls" not found
Jul 24 03:17:47.894522 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.894496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6cd4908e-c044-40f4-88b4-6606c01d3054-config-volume\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.895011 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.894990    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-trusted-ca\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.897996 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.897975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-installation-pull-secrets\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.898110 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.897978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-image-registry-private-configuration\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.898110 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.898041    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b304cd9-4445-4be1-921f-0797a250e25d-cert\") pod \"ingress-canary-prsrs\" (UID: \"8b304cd9-4445-4be1-921f-0797a250e25d\") " pod="openshift-ingress-canary/ingress-canary-prsrs"
Jul 24 03:17:47.904934 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.904900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lg95\" (UniqueName: \"kubernetes.io/projected/8b304cd9-4445-4be1-921f-0797a250e25d-kube-api-access-7lg95\") pod \"ingress-canary-prsrs\" (UID: \"8b304cd9-4445-4be1-921f-0797a250e25d\") " pod="openshift-ingress-canary/ingress-canary-prsrs"
Jul 24 03:17:47.905486 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.905461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mm5b\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-kube-api-access-8mm5b\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.905781 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.905758    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq69g\" (UniqueName: \"kubernetes.io/projected/6cd4908e-c044-40f4-88b4-6606c01d3054-kube-api-access-nq69g\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:47.905838 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.905779    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-bound-sa-token\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:47.981424 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.981387    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prsrs"
Jul 24 03:17:47.988543 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:47.988502    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wp6hh_f7984107-7911-40e1-952d-fe9561244ed4/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:48.089747 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.089688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wp6hh_f7984107-7911-40e1-952d-fe9561244ed4/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:48.163288 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.158993    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wp6hh_f7984107-7911-40e1-952d-fe9561244ed4/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:48.212414 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.212369    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prsrs"]
Jul 24 03:17:48.217509 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:17:48.217477    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b304cd9_4445_4be1_921f_0797a250e25d.slice/crio-61809f2c9cd21e9ef4897f70138ed7fdab491f8fd1184e3bd9d6ff790493e555 WatchSource:0}: Error finding container 61809f2c9cd21e9ef4897f70138ed7fdab491f8fd1184e3bd9d6ff790493e555: Status 404 returned error can't find the container with id 61809f2c9cd21e9ef4897f70138ed7fdab491f8fd1184e3bd9d6ff790493e555
Jul 24 03:17:48.297252 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.297214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:48.297437 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:48.297405    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 03:17:48.297483 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:48.297458    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert podName:efaf2452-d67f-4599-998a-4a496d2c9c07 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:49.297445119 +0000 UTC m=+36.058353578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sxk8t" (UID: "efaf2452-d67f-4599-998a-4a496d2c9c07") : secret "networking-console-plugin-cert" not found
Jul 24 03:17:48.398570 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.398482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:48.398744 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.398583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:48.398744 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:48.398664    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:17:48.398744 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:48.398676    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:17:48.398744 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:48.398690    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7df49bb-g94h2: secret "image-registry-tls" not found
Jul 24 03:17:48.398744 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:48.398744    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls podName:15f782ab-f003-4bf8-8c63-4343d1afcc61 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:49.398727667 +0000 UTC m=+36.159636126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls") pod "image-registry-d7df49bb-g94h2" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61") : secret "image-registry-tls" not found
Jul 24 03:17:48.398985 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:48.398781    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls podName:6cd4908e-c044-40f4-88b4-6606c01d3054 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:49.39876716 +0000 UTC m=+36.159675618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls") pod "dns-default-6xp58" (UID: "6cd4908e-c044-40f4-88b4-6606c01d3054") : secret "dns-default-metrics-tls" not found
Jul 24 03:17:48.636928 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.636897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:17:48.637383 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.637074    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:17:48.639345 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.639325    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 03:17:48.639345 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.639327    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frl8x\""
Jul 24 03:17:48.639509 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.639363    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 03:17:48.863991 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:48.863950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prsrs" event={"ID":"8b304cd9-4445-4be1-921f-0797a250e25d","Type":"ContainerStarted","Data":"61809f2c9cd21e9ef4897f70138ed7fdab491f8fd1184e3bd9d6ff790493e555"}
Jul 24 03:17:49.306878 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:49.306838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:49.307141 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:49.306995    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 03:17:49.307141 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:49.307055    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert podName:efaf2452-d67f-4599-998a-4a496d2c9c07 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:51.307036393 +0000 UTC m=+38.067944853 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sxk8t" (UID: "efaf2452-d67f-4599-998a-4a496d2c9c07") : secret "networking-console-plugin-cert" not found
Jul 24 03:17:49.407341 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:49.407291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:49.407566 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:49.407431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:49.407566 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:49.407457    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:17:49.407566 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:49.407530    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls podName:6cd4908e-c044-40f4-88b4-6606c01d3054 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:51.407509311 +0000 UTC m=+38.168417783 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls") pod "dns-default-6xp58" (UID: "6cd4908e-c044-40f4-88b4-6606c01d3054") : secret "dns-default-metrics-tls" not found
Jul 24 03:17:49.407566 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:49.407565    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:17:49.407779 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:49.407578    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7df49bb-g94h2: secret "image-registry-tls" not found
Jul 24 03:17:49.407779 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:49.407628    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls podName:15f782ab-f003-4bf8-8c63-4343d1afcc61 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:51.407611846 +0000 UTC m=+38.168520317 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls") pod "image-registry-d7df49bb-g94h2" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61") : secret "image-registry-tls" not found
Jul 24 03:17:51.323710 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:51.323670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:51.324418 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:51.323830    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 03:17:51.324418 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:51.323910    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert podName:efaf2452-d67f-4599-998a-4a496d2c9c07 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:55.323889274 +0000 UTC m=+42.084797732 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sxk8t" (UID: "efaf2452-d67f-4599-998a-4a496d2c9c07") : secret "networking-console-plugin-cert" not found
Jul 24 03:17:51.425042 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:51.425008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:51.425218 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:51.425069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:51.425218 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:51.425170    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:17:51.425218 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:51.425174    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:17:51.425218 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:51.425192    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7df49bb-g94h2: secret "image-registry-tls" not found
Jul 24 03:17:51.425408 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:51.425235    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls podName:6cd4908e-c044-40f4-88b4-6606c01d3054 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:55.42521769 +0000 UTC m=+42.186126147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls") pod "dns-default-6xp58" (UID: "6cd4908e-c044-40f4-88b4-6606c01d3054") : secret "dns-default-metrics-tls" not found
Jul 24 03:17:51.425408 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:51.425248    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls podName:15f782ab-f003-4bf8-8c63-4343d1afcc61 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:55.425242112 +0000 UTC m=+42.186150570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls") pod "image-registry-d7df49bb-g94h2" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61") : secret "image-registry-tls" not found
Jul 24 03:17:52.877444 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:52.877408    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c20bb78-3f2a-4df1-ab85-1f38ca5aa566" containerID="02b95b3b16df4b2c0d04fdd3e4c9e85026b520680cb62c4b23c9430b471309c4" exitCode=0
Jul 24 03:17:52.878116 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:52.877497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgmkm" event={"ID":"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566","Type":"ContainerDied","Data":"02b95b3b16df4b2c0d04fdd3e4c9e85026b520680cb62c4b23c9430b471309c4"}
Jul 24 03:17:52.878878 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:52.878862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prsrs" event={"ID":"8b304cd9-4445-4be1-921f-0797a250e25d","Type":"ContainerStarted","Data":"ed4a65bb4286254a81d393a6279f7355d3a27a81281b6c2ba08a8b39a0a75113"}
Jul 24 03:17:52.912556 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:52.912501    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-prsrs" podStartSLOduration=1.9430831130000001 podStartE2EDuration="5.912484786s" totalImagesPullingTime="3.969401673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:52.912392901 +0000 UTC m=+39.673301381" watchObservedRunningTime="2026-07-24 03:17:52.912484786 +0000 UTC m=+39.673393271"
Jul 24 03:17:53.884582 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:53.884544    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c20bb78-3f2a-4df1-ab85-1f38ca5aa566" containerID="3fd860bcb3eba60e6241457728ce6892bb766d3b2ecff5f588c826187b1c4c5a" exitCode=0
Jul 24 03:17:53.884990 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:53.884640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgmkm" event={"ID":"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566","Type":"ContainerDied","Data":"3fd860bcb3eba60e6241457728ce6892bb766d3b2ecff5f588c826187b1c4c5a"}
Jul 24 03:17:54.893236 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:54.893197    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wgmkm" event={"ID":"6c20bb78-3f2a-4df1-ab85-1f38ca5aa566","Type":"ContainerStarted","Data":"229cf9c1e29870e3e2c77a2ee9dd29ba6f1885468b1fffddbcc6cf6d7f9bebd0"}
Jul 24 03:17:54.916009 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:54.915954    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wgmkm" podStartSLOduration=8.101643403 podStartE2EDuration="41.915940137s" totalImagesPullingTime="33.814296734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:54.914535803 +0000 UTC m=+41.675444282" watchObservedRunningTime="2026-07-24 03:17:54.915940137 +0000 UTC m=+41.676848636"
Jul 24 03:17:55.357979 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:55.357937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:17:55.358154 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:55.358073    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 03:17:55.358154 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:55.358139    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert podName:efaf2452-d67f-4599-998a-4a496d2c9c07 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:03.358122378 +0000 UTC m=+50.119030837 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sxk8t" (UID: "efaf2452-d67f-4599-998a-4a496d2c9c07") : secret "networking-console-plugin-cert" not found
Jul 24 03:17:55.458995 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:55.458956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:17:55.459136 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:17:55.459013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:17:55.459136 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:55.459099    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:17:55.459136 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:55.459104    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:17:55.459136 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:55.459124    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7df49bb-g94h2: secret "image-registry-tls" not found
Jul 24 03:17:55.459297 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:55.459162    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls podName:6cd4908e-c044-40f4-88b4-6606c01d3054 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:03.459146483 +0000 UTC m=+50.220054958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls") pod "dns-default-6xp58" (UID: "6cd4908e-c044-40f4-88b4-6606c01d3054") : secret "dns-default-metrics-tls" not found
Jul 24 03:17:55.459297 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:17:55.459227    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls podName:15f782ab-f003-4bf8-8c63-4343d1afcc61 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:03.45921223 +0000 UTC m=+50.220120688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls") pod "image-registry-d7df49bb-g94h2" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61") : secret "image-registry-tls" not found
Jul 24 03:18:03.420625 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:03.420568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:18:03.421070 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:03.420729    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 03:18:03.421070 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:03.420823    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert podName:efaf2452-d67f-4599-998a-4a496d2c9c07 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:19.420806098 +0000 UTC m=+66.181714555 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sxk8t" (UID: "efaf2452-d67f-4599-998a-4a496d2c9c07") : secret "networking-console-plugin-cert" not found
Jul 24 03:18:03.521304 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:03.521234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:18:03.521515 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:03.521377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:18:03.521515 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:03.521391    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:03.521515 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:03.521463    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls podName:6cd4908e-c044-40f4-88b4-6606c01d3054 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:19.521448156 +0000 UTC m=+66.282356619 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls") pod "dns-default-6xp58" (UID: "6cd4908e-c044-40f4-88b4-6606c01d3054") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:03.521515 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:03.521499    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:03.521515 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:03.521512    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7df49bb-g94h2: secret "image-registry-tls" not found
Jul 24 03:18:03.521718 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:03.521587    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls podName:15f782ab-f003-4bf8-8c63-4343d1afcc61 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:19.521570454 +0000 UTC m=+66.282478927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls") pod "image-registry-d7df49bb-g94h2" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61") : secret "image-registry-tls" not found
Jul 24 03:18:13.866318 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:13.866285    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzp5b"
Jul 24 03:18:18.437322 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.437280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:18:18.437704 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.437354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:18:18.437704 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.437382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:18:18.439779 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.439760    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 03:18:18.440061 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.440047    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 03:18:18.440115 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.440100    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 03:18:18.448240 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:18.448224    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 03:18:18.448308 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:18.448297    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:19:22.448274252 +0000 UTC m=+129.209182723 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : secret "metrics-daemon-secret" not found
Jul 24 03:18:18.450099 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.450084    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 03:18:18.451385 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.451368    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69bd88a4-0811-4ba8-a935-20746395f8a6-original-pull-secret\") pod \"global-pull-secret-syncer-4mflr\" (UID: \"69bd88a4-0811-4ba8-a935-20746395f8a6\") " pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:18:18.461099 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.461072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbwdt\" (UniqueName: \"kubernetes.io/projected/ac48200c-f150-494e-9f56-d1fdfc8df128-kube-api-access-kbwdt\") pod \"network-check-target-qsw5d\" (UID: \"ac48200c-f150-494e-9f56-d1fdfc8df128\") " pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:18:18.581722 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.581690    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n6kfx\""
Jul 24 03:18:18.590081 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.590064    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:18:18.654325 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.654288    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4mflr"
Jul 24 03:18:18.716455 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.716372    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qsw5d"]
Jul 24 03:18:18.718904 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:18:18.718878    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac48200c_f150_494e_9f56_d1fdfc8df128.slice/crio-ae8807849c1dc6a778b973af152cf54594412b177b3c31e8f0fdf1c3ac8e6c4a WatchSource:0}: Error finding container ae8807849c1dc6a778b973af152cf54594412b177b3c31e8f0fdf1c3ac8e6c4a: Status 404 returned error can't find the container with id ae8807849c1dc6a778b973af152cf54594412b177b3c31e8f0fdf1c3ac8e6c4a
Jul 24 03:18:18.781151 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.781117    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4mflr"]
Jul 24 03:18:18.784145 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:18:18.784117    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69bd88a4_0811_4ba8_a935_20746395f8a6.slice/crio-55f698e0f4baf7e4c5f01b0c5582a88ab2ff2d96498cadfc30708a3aa6d672cc WatchSource:0}: Error finding container 55f698e0f4baf7e4c5f01b0c5582a88ab2ff2d96498cadfc30708a3aa6d672cc: Status 404 returned error can't find the container with id 55f698e0f4baf7e4c5f01b0c5582a88ab2ff2d96498cadfc30708a3aa6d672cc
Jul 24 03:18:18.949320 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.949258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4mflr" event={"ID":"69bd88a4-0811-4ba8-a935-20746395f8a6","Type":"ContainerStarted","Data":"55f698e0f4baf7e4c5f01b0c5582a88ab2ff2d96498cadfc30708a3aa6d672cc"}
Jul 24 03:18:18.950456 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:18.950432    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qsw5d" event={"ID":"ac48200c-f150-494e-9f56-d1fdfc8df128","Type":"ContainerStarted","Data":"ae8807849c1dc6a778b973af152cf54594412b177b3c31e8f0fdf1c3ac8e6c4a"}
Jul 24 03:18:19.446067 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:19.446023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:18:19.446588 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:19.446204    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 03:18:19.446588 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:19.446328    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert podName:efaf2452-d67f-4599-998a-4a496d2c9c07 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:51.446301841 +0000 UTC m=+98.207210313 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sxk8t" (UID: "efaf2452-d67f-4599-998a-4a496d2c9c07") : secret "networking-console-plugin-cert" not found
Jul 24 03:18:19.546502 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:19.546460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:18:19.546705 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:19.546525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:18:19.546705 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:19.546631    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:19.546705 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:19.546657    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7df49bb-g94h2: secret "image-registry-tls" not found
Jul 24 03:18:19.546705 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:19.546665    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:19.546855 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:19.546716    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls podName:15f782ab-f003-4bf8-8c63-4343d1afcc61 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:51.546699227 +0000 UTC m=+98.307607688 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls") pod "image-registry-d7df49bb-g94h2" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61") : secret "image-registry-tls" not found
Jul 24 03:18:19.546855 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:19.546729    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls podName:6cd4908e-c044-40f4-88b4-6606c01d3054 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:51.546723181 +0000 UTC m=+98.307631642 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls") pod "dns-default-6xp58" (UID: "6cd4908e-c044-40f4-88b4-6606c01d3054") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:23.965419 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:23.965387    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4mflr" event={"ID":"69bd88a4-0811-4ba8-a935-20746395f8a6","Type":"ContainerStarted","Data":"52cf05caef60526b70d8809439bf3242a104a13451931dfa859298057d5f68ee"}
Jul 24 03:18:23.966935 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:23.966915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qsw5d" event={"ID":"ac48200c-f150-494e-9f56-d1fdfc8df128","Type":"ContainerStarted","Data":"44da9bc8e0ee50fe7e33d79e68eb32a32e71a0b730aca07dcf3b192b6e8f1d5d"}
Jul 24 03:18:23.967063 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:23.967048    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:18:23.980688 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:23.980638    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4mflr" podStartSLOduration=65.416710657 podStartE2EDuration="1m9.980622818s" totalImagesPullingTime="4.563912161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:23.980015573 +0000 UTC m=+70.740924052" watchObservedRunningTime="2026-07-24 03:18:23.980622818 +0000 UTC m=+70.741531298"
Jul 24 03:18:23.994462 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:23.994413    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qsw5d" podStartSLOduration=66.369605715 podStartE2EDuration="1m10.99439583s" totalImagesPullingTime="4.624790115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:23.993638276 +0000 UTC m=+70.754546766" watchObservedRunningTime="2026-07-24 03:18:23.99439583 +0000 UTC m=+70.755304358"
Jul 24 03:18:51.495014 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:51.494868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:18:51.495573 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:51.495028    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 03:18:51.495573 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:51.495098    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert podName:efaf2452-d67f-4599-998a-4a496d2c9c07 nodeName:}" failed. No retries permitted until 2026-07-24 03:19:55.495081249 +0000 UTC m=+162.255989711 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sxk8t" (UID: "efaf2452-d67f-4599-998a-4a496d2c9c07") : secret "networking-console-plugin-cert" not found
Jul 24 03:18:51.595513 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:51.595475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:18:51.595681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:51.595525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:18:51.595681 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:51.595633    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:51.595681 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:51.595645    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:51.595681 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:51.595667    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7df49bb-g94h2: secret "image-registry-tls" not found
Jul 24 03:18:51.595840 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:51.595693    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls podName:6cd4908e-c044-40f4-88b4-6606c01d3054 nodeName:}" failed. No retries permitted until 2026-07-24 03:19:55.595679194 +0000 UTC m=+162.356587653 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls") pod "dns-default-6xp58" (UID: "6cd4908e-c044-40f4-88b4-6606c01d3054") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:51.595840 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:18:51.595719    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls podName:15f782ab-f003-4bf8-8c63-4343d1afcc61 nodeName:}" failed. No retries permitted until 2026-07-24 03:19:55.595701713 +0000 UTC m=+162.356610188 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls") pod "image-registry-d7df49bb-g94h2" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61") : secret "image-registry-tls" not found
Jul 24 03:18:54.970818 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:18:54.970783    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qsw5d"
Jul 24 03:19:01.064690 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.064654    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"]
Jul 24 03:19:01.068049 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.068032    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.070579 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.070553    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 03:19:01.070579 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.070565    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 24 03:19:01.071507 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.071484    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 24 03:19:01.071605 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.071547    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 24 03:19:01.071605 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.071484    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 24 03:19:01.071605 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.071491    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 03:19:01.071605 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.071579    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 03:19:01.080567 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.080548    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"]
Jul 24 03:19:01.166843 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.166804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-hub\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.166843 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.166841    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/916fcb39-f3d0-4d11-bd2f-3dfffed79593-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.167051 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.166884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.167051 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.166928    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.167051 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.166996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-ca\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.167051 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.167024    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b448\" (UniqueName: \"kubernetes.io/projected/916fcb39-f3d0-4d11-bd2f-3dfffed79593-kube-api-access-4b448\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.267422 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.267382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-ca\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.267422 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.267428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b448\" (UniqueName: \"kubernetes.io/projected/916fcb39-f3d0-4d11-bd2f-3dfffed79593-kube-api-access-4b448\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.267613 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.267487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-hub\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.267613 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.267506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/916fcb39-f3d0-4d11-bd2f-3dfffed79593-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.267613 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.267559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.267613 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.267588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.268331 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.268248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/916fcb39-f3d0-4d11-bd2f-3dfffed79593-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.270055 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.270028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-ca\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.270230 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.270210    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-hub\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.270319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.270254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.270410 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.270338    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/916fcb39-f3d0-4d11-bd2f-3dfffed79593-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.275242 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.275208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b448\" (UniqueName: \"kubernetes.io/projected/916fcb39-f3d0-4d11-bd2f-3dfffed79593-kube-api-access-4b448\") pod \"cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl\" (UID: \"916fcb39-f3d0-4d11-bd2f-3dfffed79593\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.389480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.389365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:19:01.521753 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:01.521718    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"]
Jul 24 03:19:01.525413 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:19:01.525386    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod916fcb39_f3d0_4d11_bd2f_3dfffed79593.slice/crio-820e382a6f8aa853f7e767d39c9dcdadfa329726a79fc7ac3554bf29e55058e9 WatchSource:0}: Error finding container 820e382a6f8aa853f7e767d39c9dcdadfa329726a79fc7ac3554bf29e55058e9: Status 404 returned error can't find the container with id 820e382a6f8aa853f7e767d39c9dcdadfa329726a79fc7ac3554bf29e55058e9
Jul 24 03:19:02.060622 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:02.060582    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" event={"ID":"916fcb39-f3d0-4d11-bd2f-3dfffed79593","Type":"ContainerStarted","Data":"820e382a6f8aa853f7e767d39c9dcdadfa329726a79fc7ac3554bf29e55058e9"}
Jul 24 03:19:06.074120 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:06.074076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" event={"ID":"916fcb39-f3d0-4d11-bd2f-3dfffed79593","Type":"ContainerStarted","Data":"c10bb2c2ed05ee1eb373c964d7c08441165137a6edac4a3ea3f1d5f426721e6d"}
Jul 24 03:19:06.074519 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:06.074133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" event={"ID":"916fcb39-f3d0-4d11-bd2f-3dfffed79593","Type":"ContainerStarted","Data":"a887df389582d2be5ec47f1ec174af50cd70055c1e399aee2f2d8e60659937b0"}
Jul 24 03:19:06.074519 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:06.074152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" event={"ID":"916fcb39-f3d0-4d11-bd2f-3dfffed79593","Type":"ContainerStarted","Data":"b57deddf048db8d091fc2c1bee9c9eeff92ea394344fe91c91ba836389d3e8ea"}
Jul 24 03:19:06.093256 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:06.093207    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" podStartSLOduration=1.368162241 podStartE2EDuration="5.093191618s" totalImagesPullingTime="3.725029377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:19:06.091685736 +0000 UTC m=+112.852594215" watchObservedRunningTime="2026-07-24 03:19:06.093191618 +0000 UTC m=+112.854100174"
Jul 24 03:19:22.530790 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:22.530749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:19:22.531405 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:22.530937    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 03:19:22.531405 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:22.531025    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:24.531002244 +0000 UTC m=+251.291910707 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : secret "metrics-daemon-secret" not found
Jul 24 03:19:50.644550 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:50.644484    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t" podUID="efaf2452-d67f-4599-998a-4a496d2c9c07"
Jul 24 03:19:50.664693 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:50.664662    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-d7df49bb-g94h2" podUID="15f782ab-f003-4bf8-8c63-4343d1afcc61"
Jul 24 03:19:50.706049 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:50.706018    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6xp58" podUID="6cd4908e-c044-40f4-88b4-6606c01d3054"
Jul 24 03:19:51.199562 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:51.199530    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:19:51.199757 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:51.199531    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:19:51.199757 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:51.199544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xp58"
Jul 24 03:19:51.648409 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:51.648329    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:19:55.576889 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:55.576842    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:19:55.577293 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:55.576998    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 03:19:55.577293 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:55.577077    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert podName:efaf2452-d67f-4599-998a-4a496d2c9c07 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:57.577059972 +0000 UTC m=+284.337968431 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-sxk8t" (UID: "efaf2452-d67f-4599-998a-4a496d2c9c07") : secret "networking-console-plugin-cert" not found
Jul 24 03:19:55.677842 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:55.677803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:19:55.678088 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:19:55.677897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:19:55.678088 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:55.677939    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:19:55.678088 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:55.678003    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:19:55.678088 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:55.678020    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7df49bb-g94h2: secret "image-registry-tls" not found
Jul 24 03:19:55.678088 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:55.678007    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls podName:6cd4908e-c044-40f4-88b4-6606c01d3054 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:57.67799207 +0000 UTC m=+284.438900528 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls") pod "dns-default-6xp58" (UID: "6cd4908e-c044-40f4-88b4-6606c01d3054") : secret "dns-default-metrics-tls" not found
Jul 24 03:19:55.678088 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:19:55.678090    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls podName:15f782ab-f003-4bf8-8c63-4343d1afcc61 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:57.678075362 +0000 UTC m=+284.438983826 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls") pod "image-registry-d7df49bb-g94h2" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61") : secret "image-registry-tls" not found
Jul 24 03:20:04.636926 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:20:04.636839    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:21:22.941190 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:22.941154    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-km4rg"]
Jul 24 03:21:22.944105 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:22.944084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:22.947077 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:22.947055    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 03:21:22.948472 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:22.948451    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 03:21:22.948571 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:22.948501    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 03:21:22.948802 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:22.948776    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 03:21:22.950479 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:22.950463    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fs6xn\""
Jul 24 03:21:22.961595 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:22.961571    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-km4rg"]
Jul 24 03:21:23.016663 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.016633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.016663 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.016664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.016854 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.016689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr6cn\" (UniqueName: \"kubernetes.io/projected/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-api-access-pr6cn\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.016854 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.016798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc7c47fc-3227-4aec-bf89-07c4da81121c-data-volume\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.016854 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.016822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc7c47fc-3227-4aec-bf89-07c4da81121c-crio-socket\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.117642 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.117601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc7c47fc-3227-4aec-bf89-07c4da81121c-crio-socket\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.117825 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.117665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.117825 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.117688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.117825 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.117723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr6cn\" (UniqueName: \"kubernetes.io/projected/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-api-access-pr6cn\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.117825 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.117726    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc7c47fc-3227-4aec-bf89-07c4da81121c-crio-socket\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.117825 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:23.117803    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:23.117825 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:23.117823    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:23.118064 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.117853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc7c47fc-3227-4aec-bf89-07c4da81121c-data-volume\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.118064 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:23.117882    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:23.61786115 +0000 UTC m=+250.378769623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:23.118064 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:23.117899    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:23.617889952 +0000 UTC m=+250.378798415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:23.118168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.118127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc7c47fc-3227-4aec-bf89-07c4da81121c-data-volume\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.125684 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.125663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr6cn\" (UniqueName: \"kubernetes.io/projected/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-api-access-pr6cn\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.621586 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.621533    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.621586 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:23.621587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:23.621807 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:23.621658    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:23.621807 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:23.621710    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:23.621807 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:23.621744    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:24.621723295 +0000 UTC m=+251.382631755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:23.621807 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:23.621760    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:24.621754525 +0000 UTC m=+251.382662982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:24.630483 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:24.630442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:21:24.630935 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:24.630502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:24.630935 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:24.630523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:24.630935 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:24.630591    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:24.630935 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:24.630629    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 03:21:24.630935 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:24.630650    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:24.630935 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:24.630662    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:26.630641727 +0000 UTC m=+253.391550184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:24.630935 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:24.630684    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:26.630675528 +0000 UTC m=+253.391583986 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:24.630935 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:24.630711    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs podName:622bcd14-324b-4bae-b7e1-37f16f28c417 nodeName:}" failed. No retries permitted until 2026-07-24 03:23:26.630689975 +0000 UTC m=+373.391598445 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs") pod "network-metrics-daemon-8cs8x" (UID: "622bcd14-324b-4bae-b7e1-37f16f28c417") : secret "metrics-daemon-secret" not found
Jul 24 03:21:25.560008 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:25.559980    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8b2k6_49dc3ed5-5da8-4347-b531-d650944e4ee8/dns-node-resolver/0.log"
Jul 24 03:21:26.645596 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:26.645544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:26.645596 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:26.645599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:26.646097 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:26.645662    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:26.646097 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:26.645726    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:30.645710355 +0000 UTC m=+257.406618812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:26.646097 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:26.645745    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:26.646097 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:26.645801    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:30.645785905 +0000 UTC m=+257.406694378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:26.961435 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:26.961400    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lxx2x_d03b8479-3f64-4961-8243-96e9d6f6d551/node-ca/0.log"
Jul 24 03:21:27.760435 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:27.760398    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prsrs_8b304cd9-4445-4be1-921f-0797a250e25d/serve-healthcheck-canary/0.log"
Jul 24 03:21:30.679065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:30.679025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:30.679065 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:30.679068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:30.679584 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:30.679148    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:30.679584 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:30.679175    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:30.679584 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:30.679229    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:38.679211635 +0000 UTC m=+265.440120105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:30.679584 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:30.679245    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm podName:cc7c47fc-3227-4aec-bf89-07c4da81121c nodeName:}" failed. No retries permitted until 2026-07-24 03:21:38.679237076 +0000 UTC m=+265.440145533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-km4rg" (UID: "cc7c47fc-3227-4aec-bf89-07c4da81121c") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:38.747459 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:38.747366    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:38.747459 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:38.747413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:38.747971 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:38.747951    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc7c47fc-3227-4aec-bf89-07c4da81121c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:38.749816 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:38.749798    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc7c47fc-3227-4aec-bf89-07c4da81121c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-km4rg\" (UID: \"cc7c47fc-3227-4aec-bf89-07c4da81121c\") " pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:38.853188 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:38.853138    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-km4rg"
Jul 24 03:21:38.974830 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:38.974799    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-km4rg"]
Jul 24 03:21:38.978259 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:21:38.978233    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc7c47fc_3227_4aec_bf89_07c4da81121c.slice/crio-93600d0c5fa278910a2f5fc0e8142bd2f1e3e5eb269220683d749f5b229348e2 WatchSource:0}: Error finding container 93600d0c5fa278910a2f5fc0e8142bd2f1e3e5eb269220683d749f5b229348e2: Status 404 returned error can't find the container with id 93600d0c5fa278910a2f5fc0e8142bd2f1e3e5eb269220683d749f5b229348e2
Jul 24 03:21:39.502771 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:39.502734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-km4rg" event={"ID":"cc7c47fc-3227-4aec-bf89-07c4da81121c","Type":"ContainerStarted","Data":"93600d0c5fa278910a2f5fc0e8142bd2f1e3e5eb269220683d749f5b229348e2"}
Jul 24 03:21:44.521045 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:44.520929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-km4rg" event={"ID":"cc7c47fc-3227-4aec-bf89-07c4da81121c","Type":"ContainerStarted","Data":"5679ee8b5215d03e06577556c9d65b519fed602df22b3b04018a0296851b2145"}
Jul 24 03:21:47.531782 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:47.531694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-km4rg" event={"ID":"cc7c47fc-3227-4aec-bf89-07c4da81121c","Type":"ContainerStarted","Data":"50bc68a09a06e6fd32cd1c11432670546ee09be998756408287905d356a33f1e"}
Jul 24 03:21:51.545777 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:51.545737    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-km4rg" event={"ID":"cc7c47fc-3227-4aec-bf89-07c4da81121c","Type":"ContainerStarted","Data":"fa3cb7d0dfc46062cf0a10d7ff46c68fa804cf22345a49571ed85ee19353ef7b"}
Jul 24 03:21:51.563544 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:51.563497    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-km4rg" podStartSLOduration=17.867096193 podStartE2EDuration="29.56348355s" totalImagesPullingTime="11.696387357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:21:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:51.562588588 +0000 UTC m=+278.323497065" watchObservedRunningTime="2026-07-24 03:21:51.56348355 +0000 UTC m=+278.324392027"
Jul 24 03:21:54.201026 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:54.200973    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t" podUID="efaf2452-d67f-4599-998a-4a496d2c9c07"
Jul 24 03:21:54.201026 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:54.200973    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-d7df49bb-g94h2" podUID="15f782ab-f003-4bf8-8c63-4343d1afcc61"
Jul 24 03:21:54.201026 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:54.200980    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6xp58" podUID="6cd4908e-c044-40f4-88b4-6606c01d3054"
Jul 24 03:21:54.558655 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:54.558573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:21:54.558655 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:54.558607    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xp58"
Jul 24 03:21:54.558827 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:54.558726    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:21:55.998604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:55.998569    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nnww4"]
Jul 24 03:21:56.001877 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.001856    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.004402 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.004376    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 03:21:56.004402 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.004385    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 03:21:56.004565 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.004454    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2wwgj\""
Jul 24 03:21:56.005234 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.005215    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 03:21:56.005234 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.005228    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 03:21:56.005360 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.005310    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 03:21:56.005400 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.005358    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 03:21:56.082049 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.082008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-accelerators-collector-config\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.082049 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.082048    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0220b377-f8a0-4a99-84b4-6a9f51bdf918-root\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.082319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.082073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0220b377-f8a0-4a99-84b4-6a9f51bdf918-sys\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.082319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.082122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0220b377-f8a0-4a99-84b4-6a9f51bdf918-metrics-client-ca\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.082319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.082159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkx6b\" (UniqueName: \"kubernetes.io/projected/0220b377-f8a0-4a99-84b4-6a9f51bdf918-kube-api-access-lkx6b\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.082319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.082192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-wtmp\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.082319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.082239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.082319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.082313    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-textfile\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.082558 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.082336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-tls\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.182878 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.182835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-wtmp\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183024 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.182886    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183024 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.182915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-textfile\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183024 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.182930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-tls\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183024 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.182974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-accelerators-collector-config\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183024 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.182993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0220b377-f8a0-4a99-84b4-6a9f51bdf918-root\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183340 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.183023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-wtmp\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183340 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.183079    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0220b377-f8a0-4a99-84b4-6a9f51bdf918-sys\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183340 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.183028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0220b377-f8a0-4a99-84b4-6a9f51bdf918-sys\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183340 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:56.183110    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 03:21:56.183340 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.183120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0220b377-f8a0-4a99-84b4-6a9f51bdf918-metrics-client-ca\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183340 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.183136    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0220b377-f8a0-4a99-84b4-6a9f51bdf918-root\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183340 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.183141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkx6b\" (UniqueName: \"kubernetes.io/projected/0220b377-f8a0-4a99-84b4-6a9f51bdf918-kube-api-access-lkx6b\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183340 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:21:56.183183    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-tls podName:0220b377-f8a0-4a99-84b4-6a9f51bdf918 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:56.683160463 +0000 UTC m=+283.444068933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-tls") pod "node-exporter-nnww4" (UID: "0220b377-f8a0-4a99-84b4-6a9f51bdf918") : secret "node-exporter-tls" not found
Jul 24 03:21:56.183340 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.183338    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-textfile\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183723 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.183701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-accelerators-collector-config\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.183758 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.183703    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0220b377-f8a0-4a99-84b4-6a9f51bdf918-metrics-client-ca\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.185507 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.185492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.193086 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.193062    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkx6b\" (UniqueName: \"kubernetes.io/projected/0220b377-f8a0-4a99-84b4-6a9f51bdf918-kube-api-access-lkx6b\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.444110 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.444076    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cb84d5b8-mrbgz"]
Jul 24 03:21:56.448181 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.448163    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.450787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.450761    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 03:21:56.450787 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.450778    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 24 03:21:56.450971 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.450761    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 24 03:21:56.450971 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.450942    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gpfwj\""
Jul 24 03:21:56.451048 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.451027    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 03:21:56.451098 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.451086    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 24 03:21:56.451147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.451140    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 24 03:21:56.451204 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.451188    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 24 03:21:56.455258 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.455109    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb84d5b8-mrbgz"]
Jul 24 03:21:56.485622 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.485590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmwgd\" (UniqueName: \"kubernetes.io/projected/c2c763d0-df8e-42e5-987c-9cc85f264549-kube-api-access-cmwgd\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.485801 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.485637    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-oauth-config\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.485801 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.485713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-serving-cert\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.485801 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.485736    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-oauth-serving-cert\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.485917 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.485801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-console-config\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.485917 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.485828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-service-ca\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.586736 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.586700    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-serving-cert\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.586736 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.586745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-oauth-serving-cert\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.587005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.586769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-console-config\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.587005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.586785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-service-ca\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.587005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.586801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmwgd\" (UniqueName: \"kubernetes.io/projected/c2c763d0-df8e-42e5-987c-9cc85f264549-kube-api-access-cmwgd\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.587005 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.586924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-oauth-config\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.587557 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.587536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-oauth-serving-cert\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.587657 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.587623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-console-config\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.587694 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.587623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-service-ca\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.589505 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.589482    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-serving-cert\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.589592 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.589482    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-oauth-config\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.594851 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.594828    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmwgd\" (UniqueName: \"kubernetes.io/projected/c2c763d0-df8e-42e5-987c-9cc85f264549-kube-api-access-cmwgd\") pod \"console-5cb84d5b8-mrbgz\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") " pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.687428 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.687370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-tls\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.689894 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.689875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0220b377-f8a0-4a99-84b4-6a9f51bdf918-node-exporter-tls\") pod \"node-exporter-nnww4\" (UID: \"0220b377-f8a0-4a99-84b4-6a9f51bdf918\") " pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.758123 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.758034    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:21:56.883489 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.883452    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb84d5b8-mrbgz"]
Jul 24 03:21:56.910767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:56.910736    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nnww4"
Jul 24 03:21:56.921604 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:21:56.921572    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0220b377_f8a0_4a99_84b4_6a9f51bdf918.slice/crio-faa1354fe88c78d802db927f282e6c051399b8c0a147230d1c9645f5daac8ac2 WatchSource:0}: Error finding container faa1354fe88c78d802db927f282e6c051399b8c0a147230d1c9645f5daac8ac2: Status 404 returned error can't find the container with id faa1354fe88c78d802db927f282e6c051399b8c0a147230d1c9645f5daac8ac2
Jul 24 03:21:57.571360 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.571223    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb84d5b8-mrbgz" event={"ID":"c2c763d0-df8e-42e5-987c-9cc85f264549","Type":"ContainerStarted","Data":"e5a59ee5d38a2c65b209534bc4d6f5154c1873f90633ab31df1700cf4aff36d4"}
Jul 24 03:21:57.572687 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.572620    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nnww4" event={"ID":"0220b377-f8a0-4a99-84b4-6a9f51bdf918","Type":"ContainerStarted","Data":"faa1354fe88c78d802db927f282e6c051399b8c0a147230d1c9645f5daac8ac2"}
Jul 24 03:21:57.593529 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.593447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:21:57.596657 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.596625    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efaf2452-d67f-4599-998a-4a496d2c9c07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-sxk8t\" (UID: \"efaf2452-d67f-4599-998a-4a496d2c9c07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:21:57.695608 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.694724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:21:57.695608 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.694787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:21:57.702327 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.702243    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"image-registry-d7df49bb-g94h2\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") " pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:21:57.702798 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.702752    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6cd4908e-c044-40f4-88b4-6606c01d3054-metrics-tls\") pod \"dns-default-6xp58\" (UID: \"6cd4908e-c044-40f4-88b4-6606c01d3054\") " pod="openshift-dns/dns-default-6xp58"
Jul 24 03:21:57.862826 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.862611    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fzmf8\""
Jul 24 03:21:57.862826 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.862612    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9jmtj\""
Jul 24 03:21:57.863446 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.863096    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bwrkh\""
Jul 24 03:21:57.869868 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.869790    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"
Jul 24 03:21:57.869868 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.869847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:21:57.870107 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.870082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xp58"
Jul 24 03:21:57.964397 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.962879    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-748dc4bffc-n67rs"]
Jul 24 03:21:57.975302 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.973965    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:57.978221 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.977883    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-748dc4bffc-n67rs"]
Jul 24 03:21:57.978894 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.978621    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 24 03:21:57.978894 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.978747    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 24 03:21:57.978894 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.978826    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7d2w6\""
Jul 24 03:21:57.982107 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.981951    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 24 03:21:57.982228 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.982203    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 24 03:21:57.983106 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.982748    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 24 03:21:57.983106 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:57.982971    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d5981c2sb9tr8\""
Jul 24 03:21:58.053541 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.053441    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t"]
Jul 24 03:21:58.055166 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:21:58.055134    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefaf2452_d67f_4599_998a_4a496d2c9c07.slice/crio-b2803f52be62af357e28cda2781759e3d40d54081d1d023ef9da450b7c96e8e9 WatchSource:0}: Error finding container b2803f52be62af357e28cda2781759e3d40d54081d1d023ef9da450b7c96e8e9: Status 404 returned error can't find the container with id b2803f52be62af357e28cda2781759e3d40d54081d1d023ef9da450b7c96e8e9
Jul 24 03:21:58.099078 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.099033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.099078 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.099081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-metrics-client-ca\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.099332 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.099107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-grpc-tls\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.099332 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.099145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhjdj\" (UniqueName: \"kubernetes.io/projected/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-kube-api-access-nhjdj\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.099332 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.099214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.099476 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.099337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.099476 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.099377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.099476 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.099413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-tls\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.200453 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.200414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.200607 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.200459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.200607 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.200557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-tls\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.200732 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.200697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.200804 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.200730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-metrics-client-ca\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.200804 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.200751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-grpc-tls\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.200804 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.200784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhjdj\" (UniqueName: \"kubernetes.io/projected/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-kube-api-access-nhjdj\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.200965 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.200838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.201717 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.201688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-metrics-client-ca\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.203877 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.203784    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.203877 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.203824    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.204130 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.204103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.204399 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.204361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-tls\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.204506 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.204492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-grpc-tls\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.206006 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.205967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.208427 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.208409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhjdj\" (UniqueName: \"kubernetes.io/projected/aa9ed326-99d8-44fe-93b5-9ce4f7feeae7-kube-api-access-nhjdj\") pod \"thanos-querier-748dc4bffc-n67rs\" (UID: \"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7\") " pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.285159 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.285114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6xp58"]
Jul 24 03:21:58.288481 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:21:58.288446    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd4908e_c044_40f4_88b4_6606c01d3054.slice/crio-17dab6efbac3c97eef3e5bb40aa549849523559e3eb6affaabe7a616ca00a7c2 WatchSource:0}: Error finding container 17dab6efbac3c97eef3e5bb40aa549849523559e3eb6affaabe7a616ca00a7c2: Status 404 returned error can't find the container with id 17dab6efbac3c97eef3e5bb40aa549849523559e3eb6affaabe7a616ca00a7c2
Jul 24 03:21:58.288720 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.288658    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d7df49bb-g94h2"]
Jul 24 03:21:58.291823 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:21:58.291798    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f782ab_f003_4bf8_8c63_4343d1afcc61.slice/crio-872431251f042eb18cbf04ce15fbbf426186ad750a504e92487547219845ae52 WatchSource:0}: Error finding container 872431251f042eb18cbf04ce15fbbf426186ad750a504e92487547219845ae52: Status 404 returned error can't find the container with id 872431251f042eb18cbf04ce15fbbf426186ad750a504e92487547219845ae52
Jul 24 03:21:58.306246 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.306198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:21:58.458685 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.458649    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-748dc4bffc-n67rs"]
Jul 24 03:21:58.462646 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:21:58.462615    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9ed326_99d8_44fe_93b5_9ce4f7feeae7.slice/crio-2b676889222d4f39ab3cb23c06bc11e1228a2d645603920e2b569b86177cbf42 WatchSource:0}: Error finding container 2b676889222d4f39ab3cb23c06bc11e1228a2d645603920e2b569b86177cbf42: Status 404 returned error can't find the container with id 2b676889222d4f39ab3cb23c06bc11e1228a2d645603920e2b569b86177cbf42
Jul 24 03:21:58.580540 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.580500    2577 generic.go:356] "Generic (PLEG): container finished" podID="0220b377-f8a0-4a99-84b4-6a9f51bdf918" containerID="afd236436be873fe6bb1ffc12535ae630d915bf28930f547c28172f0b76f3c69" exitCode=0
Jul 24 03:21:58.581006 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.580583    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nnww4" event={"ID":"0220b377-f8a0-4a99-84b4-6a9f51bdf918","Type":"ContainerDied","Data":"afd236436be873fe6bb1ffc12535ae630d915bf28930f547c28172f0b76f3c69"}
Jul 24 03:21:58.581981 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.581952    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t" event={"ID":"efaf2452-d67f-4599-998a-4a496d2c9c07","Type":"ContainerStarted","Data":"b2803f52be62af357e28cda2781759e3d40d54081d1d023ef9da450b7c96e8e9"}
Jul 24 03:21:58.583658 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.583631    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d7df49bb-g94h2" event={"ID":"15f782ab-f003-4bf8-8c63-4343d1afcc61","Type":"ContainerStarted","Data":"cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe"}
Jul 24 03:21:58.583759 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.583689    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:21:58.583759 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.583711    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d7df49bb-g94h2" event={"ID":"15f782ab-f003-4bf8-8c63-4343d1afcc61","Type":"ContainerStarted","Data":"872431251f042eb18cbf04ce15fbbf426186ad750a504e92487547219845ae52"}
Jul 24 03:21:58.585658 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.585584    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs" event={"ID":"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7","Type":"ContainerStarted","Data":"2b676889222d4f39ab3cb23c06bc11e1228a2d645603920e2b569b86177cbf42"}
Jul 24 03:21:58.587173 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.587146    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xp58" event={"ID":"6cd4908e-c044-40f4-88b4-6606c01d3054","Type":"ContainerStarted","Data":"17dab6efbac3c97eef3e5bb40aa549849523559e3eb6affaabe7a616ca00a7c2"}
Jul 24 03:21:58.618202 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:58.617902    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d7df49bb-g94h2" podStartSLOduration=284.617884857 podStartE2EDuration="4m44.617884857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:58.617363126 +0000 UTC m=+285.378271606" watchObservedRunningTime="2026-07-24 03:21:58.617884857 +0000 UTC m=+285.378793314"
Jul 24 03:21:59.594620 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:59.594579    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nnww4" event={"ID":"0220b377-f8a0-4a99-84b4-6a9f51bdf918","Type":"ContainerStarted","Data":"50122601d8435dc40900b6a038b5974ecb789200f9c8e83afd74dc0540dbd7c2"}
Jul 24 03:21:59.595081 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:59.594698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nnww4" event={"ID":"0220b377-f8a0-4a99-84b4-6a9f51bdf918","Type":"ContainerStarted","Data":"605d9a194308c10f7adaaa52850a79e72698add72458814d4a183b1c1653ce4d"}
Jul 24 03:21:59.614447 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:21:59.614222    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nnww4" podStartSLOduration=3.51693904 podStartE2EDuration="4.61420223s" totalImagesPullingTime="1.015385968s" totalInitContainerRuntime="81.877222ms" isStatefulPod=true podCreationTimestamp="2026-07-24 03:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:59.613092853 +0000 UTC m=+286.374001333" watchObservedRunningTime="2026-07-24 03:21:59.61420223 +0000 UTC m=+286.375110712"
Jul 24 03:22:01.390851 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:01.390783    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" podUID="916fcb39-f3d0-4d11-bd2f-3dfffed79593" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 03:22:01.604289 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:01.604227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb84d5b8-mrbgz" event={"ID":"c2c763d0-df8e-42e5-987c-9cc85f264549","Type":"ContainerStarted","Data":"666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466"}
Jul 24 03:22:01.606419 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:01.606372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xp58" event={"ID":"6cd4908e-c044-40f4-88b4-6606c01d3054","Type":"ContainerStarted","Data":"6f2c1e0ad697db1000ec4e34b45419900dc2aec4b6384657ffcbb5b2c3b48e10"}
Jul 24 03:22:01.606554 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:01.606430    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6xp58"
Jul 24 03:22:01.606554 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:01.606458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xp58" event={"ID":"6cd4908e-c044-40f4-88b4-6606c01d3054","Type":"ContainerStarted","Data":"66ab0d231acdd831d4160ccdf753683fa311e9c81e70fa1cee3fe35235be7037"}
Jul 24 03:22:01.608072 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:01.608043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t" event={"ID":"efaf2452-d67f-4599-998a-4a496d2c9c07","Type":"ContainerStarted","Data":"ce54e8b776744cd389e2cbafae3b7392f0e1d4caa1caa47274a5c7c294185549"}
Jul 24 03:22:01.621807 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:01.621712    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cb84d5b8-mrbgz" podStartSLOduration=1.585803929 podStartE2EDuration="5.621697961s" totalImagesPullingTime="4.035894032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:01.621299435 +0000 UTC m=+288.382207917" watchObservedRunningTime="2026-07-24 03:22:01.621697961 +0000 UTC m=+288.382606443"
Jul 24 03:22:01.635779 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:01.635729    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-sxk8t" podStartSLOduration=265.767313903 podStartE2EDuration="4m28.635714081s" totalImagesPullingTime="2.868400178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:01.635292233 +0000 UTC m=+288.396200714" watchObservedRunningTime="2026-07-24 03:22:01.635714081 +0000 UTC m=+288.396622560"
Jul 24 03:22:01.652071 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:01.651984    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6xp58" podStartSLOduration=252.01180574 podStartE2EDuration="4m14.65196363s" totalImagesPullingTime="2.64015789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:01.651180625 +0000 UTC m=+288.412089105" watchObservedRunningTime="2026-07-24 03:22:01.65196363 +0000 UTC m=+288.412872109"
Jul 24 03:22:02.613220 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:02.613174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs" event={"ID":"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7","Type":"ContainerStarted","Data":"1adddf509ff3ae886aad775c979147401675ddf1728465364be4a63520bbed40"}
Jul 24 03:22:02.613782 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:02.613254    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs" event={"ID":"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7","Type":"ContainerStarted","Data":"2948f6ee2df9dcaf3a30775c3ce43872d7fd579653c5ec9ac2a2adb90ce5b783"}
Jul 24 03:22:02.613782 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:02.613309    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs" event={"ID":"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7","Type":"ContainerStarted","Data":"21c1fb34477c76f331a596b73e9b3fbbc31609f671b114cfc9f268704288f5ef"}
Jul 24 03:22:03.618969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:03.618932    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs" event={"ID":"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7","Type":"ContainerStarted","Data":"5665e817c5441bc3894a608a107413eb20541264d9d6a2b9716c5d027b1059c1"}
Jul 24 03:22:03.619495 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:03.618998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs" event={"ID":"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7","Type":"ContainerStarted","Data":"a75b424257bf2700b320b1ee4b67d7da744992cc350b79d2abdfd9703081f85d"}
Jul 24 03:22:03.619495 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:03.619029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs" event={"ID":"aa9ed326-99d8-44fe-93b5-9ce4f7feeae7","Type":"ContainerStarted","Data":"7b89f42d022b8380d38780a65f292ed81d7e0cc907a8fcef2de303316ce2bc28"}
Jul 24 03:22:03.619495 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:03.619072    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:22:03.643648 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:03.643594    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs" podStartSLOduration=2.24956076 podStartE2EDuration="6.643574173s" totalImagesPullingTime="4.394013413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:03.642826569 +0000 UTC m=+290.403735049" watchObservedRunningTime="2026-07-24 03:22:03.643574173 +0000 UTC m=+290.404482654"
Jul 24 03:22:05.832942 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.832911    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f68f56c88-tmjtq"]
Jul 24 03:22:05.838114 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.838094    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:05.845926 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.845898    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 24 03:22:05.846751 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.846731    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f68f56c88-tmjtq"]
Jul 24 03:22:05.968921 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.968877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxh86\" (UniqueName: \"kubernetes.io/projected/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-kube-api-access-mxh86\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:05.968921 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.968922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-oauth-config\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:05.969147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.968948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-config\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:05.969147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.968968    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-serving-cert\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:05.969147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.968985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-service-ca\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:05.969147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.969073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-oauth-serving-cert\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:05.969147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:05.969130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-trusted-ca-bundle\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.069486 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.069449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-trusted-ca-bundle\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.069486 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.069487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxh86\" (UniqueName: \"kubernetes.io/projected/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-kube-api-access-mxh86\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.069664 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.069503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-oauth-config\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.069664 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.069530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-config\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.069664 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.069549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-serving-cert\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.069784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.069731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-service-ca\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.069834 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.069811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-oauth-serving-cert\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.070458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.070433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-oauth-serving-cert\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.070458 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.070435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-config\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.070618 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.070499    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-trusted-ca-bundle\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.070896 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.070873    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-service-ca\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.072055 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.072032    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-serving-cert\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.072149 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.072131    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-oauth-config\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.078056 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.078035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxh86\" (UniqueName: \"kubernetes.io/projected/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-kube-api-access-mxh86\") pod \"console-6f68f56c88-tmjtq\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") " pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.148227 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.148133    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:06.282057 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.282025    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f68f56c88-tmjtq"]
Jul 24 03:22:06.285093 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:22:06.285062    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a8e3be_fe4b_4d96_b704_3f346c0aec17.slice/crio-2501ffb46e20e2a13fb4b2d813ea6b3b41272904982977902ea1d7fadf9016c6 WatchSource:0}: Error finding container 2501ffb46e20e2a13fb4b2d813ea6b3b41272904982977902ea1d7fadf9016c6: Status 404 returned error can't find the container with id 2501ffb46e20e2a13fb4b2d813ea6b3b41272904982977902ea1d7fadf9016c6
Jul 24 03:22:06.630541 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.630512    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f68f56c88-tmjtq" event={"ID":"a1a8e3be-fe4b-4d96-b704-3f346c0aec17","Type":"ContainerStarted","Data":"f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118"}
Jul 24 03:22:06.630717 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.630561    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f68f56c88-tmjtq" event={"ID":"a1a8e3be-fe4b-4d96-b704-3f346c0aec17","Type":"ContainerStarted","Data":"2501ffb46e20e2a13fb4b2d813ea6b3b41272904982977902ea1d7fadf9016c6"}
Jul 24 03:22:06.647780 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.647728    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f68f56c88-tmjtq" podStartSLOduration=1.647708824 podStartE2EDuration="1.647708824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:22:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:06.647552194 +0000 UTC m=+293.408460689" watchObservedRunningTime="2026-07-24 03:22:06.647708824 +0000 UTC m=+293.408617306"
Jul 24 03:22:06.758780 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.758746    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:22:06.758948 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.758838    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:22:06.759876 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.759847    2577 patch_prober.go:28] interesting pod/console-5cb84d5b8-mrbgz container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.12:8443/health\": dial tcp 10.132.0.12:8443: connect: connection refused" start-of-body=
Jul 24 03:22:06.759998 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:06.759912    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5cb84d5b8-mrbgz" podUID="c2c763d0-df8e-42e5-987c-9cc85f264549" containerName="console" probeResult="failure" output="Get \"https://10.132.0.12:8443/health\": dial tcp 10.132.0.12:8443: connect: connection refused"
Jul 24 03:22:07.638030 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:22:07.637986    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8cs8x" podUID="622bcd14-324b-4bae-b7e1-37f16f28c417"
Jul 24 03:22:09.630080 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:09.630051    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-748dc4bffc-n67rs"
Jul 24 03:22:11.391359 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:11.391317    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" podUID="916fcb39-f3d0-4d11-bd2f-3dfffed79593" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 03:22:11.616043 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:11.616006    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6xp58"
Jul 24 03:22:13.556099 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:13.556072    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 03:22:16.148437 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:16.148397    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:16.247112 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:16.148463    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:16.247112 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:16.149843    2577 patch_prober.go:28] interesting pod/console-6f68f56c88-tmjtq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.14:8443/health\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body=
Jul 24 03:22:16.247112 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:16.149902    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6f68f56c88-tmjtq" podUID="a1a8e3be-fe4b-4d96-b704-3f346c0aec17" containerName="console" probeResult="failure" output="Get \"https://10.132.0.14:8443/health\": dial tcp 10.132.0.14:8443: connect: connection refused"
Jul 24 03:22:16.762505 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:16.762476    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:22:16.766258 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:16.766228    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:22:17.874743 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:17.874707    2577 patch_prober.go:28] interesting pod/image-registry-d7df49bb-g94h2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 03:22:17.875137 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:17.874773    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-d7df49bb-g94h2" podUID="15f782ab-f003-4bf8-8c63-4343d1afcc61" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 03:22:19.030878 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:19.030842    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d7df49bb-g94h2"]
Jul 24 03:22:19.035338 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:19.035312    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:22:19.637387 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:19.637351    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:22:21.391229 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:21.391188    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" podUID="916fcb39-f3d0-4d11-bd2f-3dfffed79593" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 03:22:21.391640 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:21.391286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl"
Jul 24 03:22:21.391932 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:21.391902    2577 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"c10bb2c2ed05ee1eb373c964d7c08441165137a6edac4a3ea3f1d5f426721e6d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 24 03:22:21.391970 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:21.391957    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" podUID="916fcb39-f3d0-4d11-bd2f-3dfffed79593" containerName="service-proxy" containerID="cri-o://c10bb2c2ed05ee1eb373c964d7c08441165137a6edac4a3ea3f1d5f426721e6d" gracePeriod=30
Jul 24 03:22:21.683776 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:21.683743    2577 generic.go:356] "Generic (PLEG): container finished" podID="916fcb39-f3d0-4d11-bd2f-3dfffed79593" containerID="c10bb2c2ed05ee1eb373c964d7c08441165137a6edac4a3ea3f1d5f426721e6d" exitCode=2
Jul 24 03:22:21.683965 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:21.683823    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" event={"ID":"916fcb39-f3d0-4d11-bd2f-3dfffed79593","Type":"ContainerDied","Data":"c10bb2c2ed05ee1eb373c964d7c08441165137a6edac4a3ea3f1d5f426721e6d"}
Jul 24 03:22:21.683965 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:21.683879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d5fd4fb9f-ckwwl" event={"ID":"916fcb39-f3d0-4d11-bd2f-3dfffed79593","Type":"ContainerStarted","Data":"2c456bedb08abc513cf5a5aa05a44b8862c545b1fee606c507f9ee9376fd23ae"}
Jul 24 03:22:26.152433 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:26.152397    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:26.156495 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:26.156469    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:22:26.214862 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:26.214826    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb84d5b8-mrbgz"]
Jul 24 03:22:33.963011 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:33.962981    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xp58_6cd4908e-c044-40f4-88b4-6606c01d3054/dns/0.log"
Jul 24 03:22:33.968487 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:33.968464    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xp58_6cd4908e-c044-40f4-88b4-6606c01d3054/kube-rbac-proxy/0.log"
Jul 24 03:22:34.053709 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:34.053681    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8b2k6_49dc3ed5-5da8-4347-b531-d650944e4ee8/dns-node-resolver/0.log"
Jul 24 03:22:44.050530 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.050469    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d7df49bb-g94h2" podUID="15f782ab-f003-4bf8-8c63-4343d1afcc61" containerName="registry" containerID="cri-o://cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe" gracePeriod=30
Jul 24 03:22:44.340260 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.340232    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:22:44.404048 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404010    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15f782ab-f003-4bf8-8c63-4343d1afcc61-ca-trust-extracted\") pod \"15f782ab-f003-4bf8-8c63-4343d1afcc61\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") "
Jul 24 03:22:44.404048 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404052    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-trusted-ca\") pod \"15f782ab-f003-4bf8-8c63-4343d1afcc61\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") "
Jul 24 03:22:44.404354 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404083    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-bound-sa-token\") pod \"15f782ab-f003-4bf8-8c63-4343d1afcc61\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") "
Jul 24 03:22:44.404354 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404111    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") pod \"15f782ab-f003-4bf8-8c63-4343d1afcc61\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") "
Jul 24 03:22:44.404354 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404142    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-image-registry-private-configuration\") pod \"15f782ab-f003-4bf8-8c63-4343d1afcc61\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") "
Jul 24 03:22:44.404354 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404163    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-installation-pull-secrets\") pod \"15f782ab-f003-4bf8-8c63-4343d1afcc61\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") "
Jul 24 03:22:44.404354 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404193    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-certificates\") pod \"15f782ab-f003-4bf8-8c63-4343d1afcc61\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") "
Jul 24 03:22:44.404627 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404569    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "15f782ab-f003-4bf8-8c63-4343d1afcc61" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:22:44.404627 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "15f782ab-f003-4bf8-8c63-4343d1afcc61" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:22:44.404744 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.404628    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mm5b\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-kube-api-access-8mm5b\") pod \"15f782ab-f003-4bf8-8c63-4343d1afcc61\" (UID: \"15f782ab-f003-4bf8-8c63-4343d1afcc61\") "
Jul 24 03:22:44.405098 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.405071    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-certificates\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:44.405224 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.405101    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15f782ab-f003-4bf8-8c63-4343d1afcc61-trusted-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:44.406826 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.406789    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "15f782ab-f003-4bf8-8c63-4343d1afcc61" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:22:44.407488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.407460    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "15f782ab-f003-4bf8-8c63-4343d1afcc61" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:22:44.407581 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.407511    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "15f782ab-f003-4bf8-8c63-4343d1afcc61" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:22:44.407581 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.407511    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "15f782ab-f003-4bf8-8c63-4343d1afcc61" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:22:44.407838 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.407815    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-kube-api-access-8mm5b" (OuterVolumeSpecName: "kube-api-access-8mm5b") pod "15f782ab-f003-4bf8-8c63-4343d1afcc61" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61"). InnerVolumeSpecName "kube-api-access-8mm5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:22:44.416152 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.416115    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15f782ab-f003-4bf8-8c63-4343d1afcc61-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "15f782ab-f003-4bf8-8c63-4343d1afcc61" (UID: "15f782ab-f003-4bf8-8c63-4343d1afcc61"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:22:44.505642 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.505607    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-bound-sa-token\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:44.505642 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.505638    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-registry-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:44.505642 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.505647    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-image-registry-private-configuration\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:44.505879 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.505657    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15f782ab-f003-4bf8-8c63-4343d1afcc61-installation-pull-secrets\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:44.505879 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.505667    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mm5b\" (UniqueName: \"kubernetes.io/projected/15f782ab-f003-4bf8-8c63-4343d1afcc61-kube-api-access-8mm5b\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:44.505879 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.505675    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15f782ab-f003-4bf8-8c63-4343d1afcc61-ca-trust-extracted\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:44.762853 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.762818    2577 generic.go:356] "Generic (PLEG): container finished" podID="15f782ab-f003-4bf8-8c63-4343d1afcc61" containerID="cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe" exitCode=0
Jul 24 03:22:44.763056 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.762880    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7df49bb-g94h2"
Jul 24 03:22:44.763056 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.762886    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d7df49bb-g94h2" event={"ID":"15f782ab-f003-4bf8-8c63-4343d1afcc61","Type":"ContainerDied","Data":"cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe"}
Jul 24 03:22:44.763056 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.762930    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d7df49bb-g94h2" event={"ID":"15f782ab-f003-4bf8-8c63-4343d1afcc61","Type":"ContainerDied","Data":"872431251f042eb18cbf04ce15fbbf426186ad750a504e92487547219845ae52"}
Jul 24 03:22:44.763056 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.762949    2577 scope.go:117] "RemoveContainer" containerID="cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe"
Jul 24 03:22:44.772996 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.772971    2577 scope.go:117] "RemoveContainer" containerID="cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe"
Jul 24 03:22:44.773320 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:22:44.773298    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe\": container with ID starting with cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe not found: ID does not exist" containerID="cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe"
Jul 24 03:22:44.773398 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.773330    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe"} err="failed to get container status \"cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe\": rpc error: code = NotFound desc = could not find container \"cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe\": container with ID starting with cee3ceaf6c5f8d5be47efcba2a2841e649d1cefe59fbc2c972a18f11eefbc6fe not found: ID does not exist"
Jul 24 03:22:44.784194 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.784162    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d7df49bb-g94h2"]
Jul 24 03:22:44.788094 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:44.788068    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d7df49bb-g94h2"]
Jul 24 03:22:45.644535 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:45.644494    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15f782ab-f003-4bf8-8c63-4343d1afcc61" path="/var/lib/kubelet/pods/15f782ab-f003-4bf8-8c63-4343d1afcc61/volumes"
Jul 24 03:22:51.237925 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.237863    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cb84d5b8-mrbgz" podUID="c2c763d0-df8e-42e5-987c-9cc85f264549" containerName="console" containerID="cri-o://666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466" gracePeriod=15
Jul 24 03:22:51.476535 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.476508    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:22:51.569048 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.568958    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-serving-cert\") pod \"c2c763d0-df8e-42e5-987c-9cc85f264549\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") "
Jul 24 03:22:51.569048 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.569001    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmwgd\" (UniqueName: \"kubernetes.io/projected/c2c763d0-df8e-42e5-987c-9cc85f264549-kube-api-access-cmwgd\") pod \"c2c763d0-df8e-42e5-987c-9cc85f264549\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") "
Jul 24 03:22:51.569048 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.569024    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-console-config\") pod \"c2c763d0-df8e-42e5-987c-9cc85f264549\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") "
Jul 24 03:22:51.569048 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.569047    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-service-ca\") pod \"c2c763d0-df8e-42e5-987c-9cc85f264549\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") "
Jul 24 03:22:51.569390 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.569070    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-oauth-serving-cert\") pod \"c2c763d0-df8e-42e5-987c-9cc85f264549\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") "
Jul 24 03:22:51.569390 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.569132    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-oauth-config\") pod \"c2c763d0-df8e-42e5-987c-9cc85f264549\" (UID: \"c2c763d0-df8e-42e5-987c-9cc85f264549\") "
Jul 24 03:22:51.569528 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.569504    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-console-config" (OuterVolumeSpecName: "console-config") pod "c2c763d0-df8e-42e5-987c-9cc85f264549" (UID: "c2c763d0-df8e-42e5-987c-9cc85f264549"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:22:51.569578 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.569520    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-service-ca" (OuterVolumeSpecName: "service-ca") pod "c2c763d0-df8e-42e5-987c-9cc85f264549" (UID: "c2c763d0-df8e-42e5-987c-9cc85f264549"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:22:51.569578 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.569524    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c2c763d0-df8e-42e5-987c-9cc85f264549" (UID: "c2c763d0-df8e-42e5-987c-9cc85f264549"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:22:51.571564 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.571539    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2c763d0-df8e-42e5-987c-9cc85f264549-kube-api-access-cmwgd" (OuterVolumeSpecName: "kube-api-access-cmwgd") pod "c2c763d0-df8e-42e5-987c-9cc85f264549" (UID: "c2c763d0-df8e-42e5-987c-9cc85f264549"). InnerVolumeSpecName "kube-api-access-cmwgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:22:51.571659 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.571584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c2c763d0-df8e-42e5-987c-9cc85f264549" (UID: "c2c763d0-df8e-42e5-987c-9cc85f264549"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:22:51.571659 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.571604    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c2c763d0-df8e-42e5-987c-9cc85f264549" (UID: "c2c763d0-df8e-42e5-987c-9cc85f264549"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:22:51.670131 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.670091    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-serving-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:51.670131 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.670124    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmwgd\" (UniqueName: \"kubernetes.io/projected/c2c763d0-df8e-42e5-987c-9cc85f264549-kube-api-access-cmwgd\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:51.670131 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.670135    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-console-config\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:51.670391 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.670144    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:51.670391 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.670153    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2c763d0-df8e-42e5-987c-9cc85f264549-oauth-serving-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:51.670391 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.670163    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2c763d0-df8e-42e5-987c-9cc85f264549-console-oauth-config\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:22:51.789257 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.789219    2577 generic.go:356] "Generic (PLEG): container finished" podID="c2c763d0-df8e-42e5-987c-9cc85f264549" containerID="666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466" exitCode=0
Jul 24 03:22:51.789426 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.789312    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb84d5b8-mrbgz"
Jul 24 03:22:51.789426 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.789311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb84d5b8-mrbgz" event={"ID":"c2c763d0-df8e-42e5-987c-9cc85f264549","Type":"ContainerDied","Data":"666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466"}
Jul 24 03:22:51.789426 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.789376    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb84d5b8-mrbgz" event={"ID":"c2c763d0-df8e-42e5-987c-9cc85f264549","Type":"ContainerDied","Data":"e5a59ee5d38a2c65b209534bc4d6f5154c1873f90633ab31df1700cf4aff36d4"}
Jul 24 03:22:51.789426 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.789404    2577 scope.go:117] "RemoveContainer" containerID="666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466"
Jul 24 03:22:51.799064 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.799009    2577 scope.go:117] "RemoveContainer" containerID="666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466"
Jul 24 03:22:51.799389 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:22:51.799366    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466\": container with ID starting with 666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466 not found: ID does not exist" containerID="666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466"
Jul 24 03:22:51.799475 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.799397    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466"} err="failed to get container status \"666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466\": rpc error: code = NotFound desc = could not find container \"666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466\": container with ID starting with 666bacc040c2bafc7412cc14fb0dfdbbf344b1a82ca971a082c055f29562d466 not found: ID does not exist"
Jul 24 03:22:51.804404 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.804376    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb84d5b8-mrbgz"]
Jul 24 03:22:51.808197 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:51.808172    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cb84d5b8-mrbgz"]
Jul 24 03:22:53.641524 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:22:53.641483    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2c763d0-df8e-42e5-987c-9cc85f264549" path="/var/lib/kubelet/pods/c2c763d0-df8e-42e5-987c-9cc85f264549/volumes"
Jul 24 03:23:18.173098 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.173061    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 03:23:18.173552 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.173328    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15f782ab-f003-4bf8-8c63-4343d1afcc61" containerName="registry"
Jul 24 03:23:18.173552 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.173339    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="15f782ab-f003-4bf8-8c63-4343d1afcc61" containerName="registry"
Jul 24 03:23:18.173552 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.173347    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2c763d0-df8e-42e5-987c-9cc85f264549" containerName="console"
Jul 24 03:23:18.173552 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.173352    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2c763d0-df8e-42e5-987c-9cc85f264549" containerName="console"
Jul 24 03:23:18.173552 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.173397    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2c763d0-df8e-42e5-987c-9cc85f264549" containerName="console"
Jul 24 03:23:18.173552 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.173405    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="15f782ab-f003-4bf8-8c63-4343d1afcc61" containerName="registry"
Jul 24 03:23:18.178171 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.178154    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.180441 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.180418    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 03:23:18.180765 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.180741    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 03:23:18.180849 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.180839    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 03:23:18.180925 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.180909    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 03:23:18.181159 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.181136    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 03:23:18.181230 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.181150    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 03:23:18.181230 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.181175    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-58l9w\""
Jul 24 03:23:18.181230 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.181187    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 03:23:18.181399 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.181227    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 03:23:18.186564 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.186547    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 03:23:18.190351 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.190330    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 03:23:18.279145 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279341 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a9181725-f359-4f2f-970e-08ac5be9379e-config-out\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279341 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a9181725-f359-4f2f-970e-08ac5be9379e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279341 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279341 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a9181725-f359-4f2f-970e-08ac5be9379e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279341 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9181725-f359-4f2f-970e-08ac5be9379e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279341 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279341    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-config-volume\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279529 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279359    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279529 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-web-config\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279529 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279529 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9181725-f359-4f2f-970e-08ac5be9379e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279529 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j724q\" (UniqueName: \"kubernetes.io/projected/a9181725-f359-4f2f-970e-08ac5be9379e-kube-api-access-j724q\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.279529 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.279502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380033 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.379996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9181725-f359-4f2f-970e-08ac5be9379e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380033 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-config-volume\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380339 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380339 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-web-config\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380339 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380339 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9181725-f359-4f2f-970e-08ac5be9379e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380339 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j724q\" (UniqueName: \"kubernetes.io/projected/a9181725-f359-4f2f-970e-08ac5be9379e-kube-api-access-j724q\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380600 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380600 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380600 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a9181725-f359-4f2f-970e-08ac5be9379e-config-out\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380600 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a9181725-f359-4f2f-970e-08ac5be9379e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380600 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.380600 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.380588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a9181725-f359-4f2f-970e-08ac5be9379e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.381111 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.381088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9181725-f359-4f2f-970e-08ac5be9379e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.381915 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.381592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9181725-f359-4f2f-970e-08ac5be9379e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.382165 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.382146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a9181725-f359-4f2f-970e-08ac5be9379e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.383499 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.383462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.383661 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.383609    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-web-config\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.383850 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.383808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.383989 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.383875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.384221 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.384202    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-config-volume\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.384444 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.384421    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.384444 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.384435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a9181725-f359-4f2f-970e-08ac5be9379e-config-out\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.384732 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.384713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a9181725-f359-4f2f-970e-08ac5be9379e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.385638 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.385615    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a9181725-f359-4f2f-970e-08ac5be9379e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.388633 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.388613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j724q\" (UniqueName: \"kubernetes.io/projected/a9181725-f359-4f2f-970e-08ac5be9379e-kube-api-access-j724q\") pod \"alertmanager-main-0\" (UID: \"a9181725-f359-4f2f-970e-08ac5be9379e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.487760 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.487673    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.624315 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.624291    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 03:23:18.627024 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:23:18.626994    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9181725_f359_4f2f_970e_08ac5be9379e.slice/crio-12f81d57e45c52d57f43ca7ecd2d8b49531fc3a172c7e170c2024ed835963e84 WatchSource:0}: Error finding container 12f81d57e45c52d57f43ca7ecd2d8b49531fc3a172c7e170c2024ed835963e84: Status 404 returned error can't find the container with id 12f81d57e45c52d57f43ca7ecd2d8b49531fc3a172c7e170c2024ed835963e84
Jul 24 03:23:18.628910 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.628893    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 03:23:18.882236 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:18.882146    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9181725-f359-4f2f-970e-08ac5be9379e","Type":"ContainerStarted","Data":"12f81d57e45c52d57f43ca7ecd2d8b49531fc3a172c7e170c2024ed835963e84"}
Jul 24 03:23:19.887145 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:19.887114    2577 generic.go:356] "Generic (PLEG): container finished" podID="a9181725-f359-4f2f-970e-08ac5be9379e" containerID="6dd929139646330dc08a51298bf9720c742d0cdcec7026d237e0e0e7182d181f" exitCode=0
Jul 24 03:23:19.887562 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:19.887176    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9181725-f359-4f2f-970e-08ac5be9379e","Type":"ContainerDied","Data":"6dd929139646330dc08a51298bf9720c742d0cdcec7026d237e0e0e7182d181f"}
Jul 24 03:23:21.899696 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:21.899602    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9181725-f359-4f2f-970e-08ac5be9379e","Type":"ContainerStarted","Data":"75f13b45f118adfc6080ce280108c401e10aad5bd3f12adfbac95b71fac9dc34"}
Jul 24 03:23:21.899696 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:21.899681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9181725-f359-4f2f-970e-08ac5be9379e","Type":"ContainerStarted","Data":"6a57d4cb412c736499a56fe35e6b9a237dba67154bea115b799a5fe2719131ac"}
Jul 24 03:23:21.900200 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:21.899727    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9181725-f359-4f2f-970e-08ac5be9379e","Type":"ContainerStarted","Data":"291677b4a18fccc712bd479360bc60f2626bacc93efa470d31054602a0f440d4"}
Jul 24 03:23:21.900200 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:21.899764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9181725-f359-4f2f-970e-08ac5be9379e","Type":"ContainerStarted","Data":"d541e4b7e8eea9011f1c7657daf8e03e91b15e54faf8d0e8bd564ea7ee9c2391"}
Jul 24 03:23:21.900200 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:21.899801    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9181725-f359-4f2f-970e-08ac5be9379e","Type":"ContainerStarted","Data":"4b6536ef821a8d84629fc0857cc67bd7166dd5b8a34dacc4788b4b7e7bbe15b5"}
Jul 24 03:23:21.900200 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:21.899850    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a9181725-f359-4f2f-970e-08ac5be9379e","Type":"ContainerStarted","Data":"a446b5544d34f898447dad0c05f78176674b159d724e4a01be64cada3494f814"}
Jul 24 03:23:21.935822 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:21.935769    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.496437635 podStartE2EDuration="3.935753293s" totalImagesPullingTime="2.423292128s" totalInitContainerRuntime="16.02353ms" isStatefulPod=false podCreationTimestamp="2026-07-24 03:23:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:23:21.934645005 +0000 UTC m=+368.695553486" watchObservedRunningTime="2026-07-24 03:23:21.935753293 +0000 UTC m=+368.696661772"
Jul 24 03:23:25.274834 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.274794    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78dfb98c4c-82lct"]
Jul 24 03:23:25.278040 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.278024    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.287107 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.287083    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78dfb98c4c-82lct"]
Jul 24 03:23:25.343405 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.343346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-oauth-config\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.343405 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.343403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6kqb\" (UniqueName: \"kubernetes.io/projected/b22a9df3-317a-48c7-ad80-accb5c59b4bf-kube-api-access-m6kqb\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.343610 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.343425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-oauth-serving-cert\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.343610 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.343474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-config\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.343610 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.343494    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-serving-cert\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.343610 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.343533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-trusted-ca-bundle\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.343610 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.343552    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-service-ca\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.444498 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.444462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-oauth-config\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.444498 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.444506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6kqb\" (UniqueName: \"kubernetes.io/projected/b22a9df3-317a-48c7-ad80-accb5c59b4bf-kube-api-access-m6kqb\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.444745 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.444522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-oauth-serving-cert\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.444745 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.444577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-config\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.444745 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.444597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-serving-cert\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.444745 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.444622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-trusted-ca-bundle\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.444745 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.444646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-service-ca\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.445491 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.445465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-service-ca\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.445621 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.445466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-config\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.445621 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.445579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-oauth-serving-cert\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.445621 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.445601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-trusted-ca-bundle\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.447355 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.447328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-oauth-config\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.447441 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.447328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-serving-cert\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.452661 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.452637    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6kqb\" (UniqueName: \"kubernetes.io/projected/b22a9df3-317a-48c7-ad80-accb5c59b4bf-kube-api-access-m6kqb\") pod \"console-78dfb98c4c-82lct\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") " pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.588505 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.588417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:25.716016 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.715903    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78dfb98c4c-82lct"]
Jul 24 03:23:25.718651 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:23:25.718620    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22a9df3_317a_48c7_ad80_accb5c59b4bf.slice/crio-af668e5fb7dc5dde3bd71b23ba3367c991bf2704ca234b8a320e95db2dfe7880 WatchSource:0}: Error finding container af668e5fb7dc5dde3bd71b23ba3367c991bf2704ca234b8a320e95db2dfe7880: Status 404 returned error can't find the container with id af668e5fb7dc5dde3bd71b23ba3367c991bf2704ca234b8a320e95db2dfe7880
Jul 24 03:23:25.916143 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.916109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dfb98c4c-82lct" event={"ID":"b22a9df3-317a-48c7-ad80-accb5c59b4bf","Type":"ContainerStarted","Data":"fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382"}
Jul 24 03:23:25.916324 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.916170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dfb98c4c-82lct" event={"ID":"b22a9df3-317a-48c7-ad80-accb5c59b4bf","Type":"ContainerStarted","Data":"af668e5fb7dc5dde3bd71b23ba3367c991bf2704ca234b8a320e95db2dfe7880"}
Jul 24 03:23:25.933116 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:25.933067    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78dfb98c4c-82lct" podStartSLOduration=0.93305302 podStartE2EDuration="933.05302ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:23:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:23:25.93170027 +0000 UTC m=+372.692608750" watchObservedRunningTime="2026-07-24 03:23:25.93305302 +0000 UTC m=+372.693961500"
Jul 24 03:23:26.655620 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:26.655570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:23:26.658190 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:26.658164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622bcd14-324b-4bae-b7e1-37f16f28c417-metrics-certs\") pod \"network-metrics-daemon-8cs8x\" (UID: \"622bcd14-324b-4bae-b7e1-37f16f28c417\") " pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:23:26.840672 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:26.840642    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frl8x\""
Jul 24 03:23:26.848632 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:26.848600    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8cs8x"
Jul 24 03:23:26.975014 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:26.974994    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8cs8x"]
Jul 24 03:23:26.977511 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:23:26.977482    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622bcd14_324b_4bae_b7e1_37f16f28c417.slice/crio-84feb27b9fc624b7d2aae90408fc80052b7f2d21320a918557a9808f61b98442 WatchSource:0}: Error finding container 84feb27b9fc624b7d2aae90408fc80052b7f2d21320a918557a9808f61b98442: Status 404 returned error can't find the container with id 84feb27b9fc624b7d2aae90408fc80052b7f2d21320a918557a9808f61b98442
Jul 24 03:23:27.926289 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:27.926236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8cs8x" event={"ID":"622bcd14-324b-4bae-b7e1-37f16f28c417","Type":"ContainerStarted","Data":"84feb27b9fc624b7d2aae90408fc80052b7f2d21320a918557a9808f61b98442"}
Jul 24 03:23:28.931884 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:28.931834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8cs8x" event={"ID":"622bcd14-324b-4bae-b7e1-37f16f28c417","Type":"ContainerStarted","Data":"baca3ce3a5a1882762cf9fbf85b4b078cb8e153061693f9dc450e2fbd20e202c"}
Jul 24 03:23:28.931884 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:28.931887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8cs8x" event={"ID":"622bcd14-324b-4bae-b7e1-37f16f28c417","Type":"ContainerStarted","Data":"7dab37f2f17a44faec31f294ffe7469434c81147443bf67cc5bcd48628217d1f"}
Jul 24 03:23:28.951316 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:28.951256    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8cs8x" podStartSLOduration=375.055336804 podStartE2EDuration="6m15.951241407s" totalImagesPullingTime="895.904603ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:23:28.94929342 +0000 UTC m=+375.710201898" watchObservedRunningTime="2026-07-24 03:23:28.951241407 +0000 UTC m=+375.712149887"
Jul 24 03:23:35.589057 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:35.589005    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:35.589486 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:35.589086    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:35.593755 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:35.593732    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:35.960658 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:35.960630    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:23:36.005728 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:23:36.005693    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f68f56c88-tmjtq"]
Jul 24 03:24:01.025131 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.025085    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f68f56c88-tmjtq" podUID="a1a8e3be-fe4b-4d96-b704-3f346c0aec17" containerName="console" containerID="cri-o://f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118" gracePeriod=15
Jul 24 03:24:01.276228 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.276169    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:24:01.443734 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.443698    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-service-ca\") pod \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") "
Jul 24 03:24:01.443734 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.443737    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-trusted-ca-bundle\") pod \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") "
Jul 24 03:24:01.444003 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.443788    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-oauth-config\") pod \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") "
Jul 24 03:24:01.444003 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.443805    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-config\") pod \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") "
Jul 24 03:24:01.444003 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.443920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-serving-cert\") pod \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") "
Jul 24 03:24:01.444003 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.443972    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-oauth-serving-cert\") pod \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") "
Jul 24 03:24:01.444208 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.444009    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxh86\" (UniqueName: \"kubernetes.io/projected/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-kube-api-access-mxh86\") pod \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\" (UID: \"a1a8e3be-fe4b-4d96-b704-3f346c0aec17\") "
Jul 24 03:24:01.444288 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.444192    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-service-ca" (OuterVolumeSpecName: "service-ca") pod "a1a8e3be-fe4b-4d96-b704-3f346c0aec17" (UID: "a1a8e3be-fe4b-4d96-b704-3f346c0aec17"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:24:01.444288 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.444208    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-config" (OuterVolumeSpecName: "console-config") pod "a1a8e3be-fe4b-4d96-b704-3f346c0aec17" (UID: "a1a8e3be-fe4b-4d96-b704-3f346c0aec17"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:24:01.444400 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.444288    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a1a8e3be-fe4b-4d96-b704-3f346c0aec17" (UID: "a1a8e3be-fe4b-4d96-b704-3f346c0aec17"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:24:01.444400 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.444338    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:24:01.444400 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.444355    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-config\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:24:01.444400 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.444372    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a1a8e3be-fe4b-4d96-b704-3f346c0aec17" (UID: "a1a8e3be-fe4b-4d96-b704-3f346c0aec17"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:24:01.446155 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.446127    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a1a8e3be-fe4b-4d96-b704-3f346c0aec17" (UID: "a1a8e3be-fe4b-4d96-b704-3f346c0aec17"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:24:01.446291 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.446162    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a1a8e3be-fe4b-4d96-b704-3f346c0aec17" (UID: "a1a8e3be-fe4b-4d96-b704-3f346c0aec17"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:24:01.446291 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.446230    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-kube-api-access-mxh86" (OuterVolumeSpecName: "kube-api-access-mxh86") pod "a1a8e3be-fe4b-4d96-b704-3f346c0aec17" (UID: "a1a8e3be-fe4b-4d96-b704-3f346c0aec17"). InnerVolumeSpecName "kube-api-access-mxh86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:24:01.545389 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.545305    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-oauth-serving-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:24:01.545389 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.545331    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxh86\" (UniqueName: \"kubernetes.io/projected/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-kube-api-access-mxh86\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:24:01.545389 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.545340    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-trusted-ca-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:24:01.545389 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.545348    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-oauth-config\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:24:01.545389 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:01.545357    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1a8e3be-fe4b-4d96-b704-3f346c0aec17-console-serving-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:24:02.052087 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:02.052057    2577 generic.go:356] "Generic (PLEG): container finished" podID="a1a8e3be-fe4b-4d96-b704-3f346c0aec17" containerID="f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118" exitCode=0
Jul 24 03:24:02.052586 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:02.052094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f68f56c88-tmjtq" event={"ID":"a1a8e3be-fe4b-4d96-b704-3f346c0aec17","Type":"ContainerDied","Data":"f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118"}
Jul 24 03:24:02.052586 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:02.052106    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f68f56c88-tmjtq"
Jul 24 03:24:02.052586 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:02.052128    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f68f56c88-tmjtq" event={"ID":"a1a8e3be-fe4b-4d96-b704-3f346c0aec17","Type":"ContainerDied","Data":"2501ffb46e20e2a13fb4b2d813ea6b3b41272904982977902ea1d7fadf9016c6"}
Jul 24 03:24:02.052586 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:02.052147    2577 scope.go:117] "RemoveContainer" containerID="f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118"
Jul 24 03:24:02.065069 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:02.065043    2577 scope.go:117] "RemoveContainer" containerID="f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118"
Jul 24 03:24:02.065367 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:24:02.065347    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118\": container with ID starting with f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118 not found: ID does not exist" containerID="f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118"
Jul 24 03:24:02.065421 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:02.065376    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118"} err="failed to get container status \"f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118\": rpc error: code = NotFound desc = could not find container \"f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118\": container with ID starting with f47a001e90a66f5bd87f58a7952c7086ff638463d1681825dbf67d85e2888118 not found: ID does not exist"
Jul 24 03:24:02.071333 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:02.071312    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f68f56c88-tmjtq"]
Jul 24 03:24:02.075646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:02.075624    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f68f56c88-tmjtq"]
Jul 24 03:24:03.641078 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:03.641044    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1a8e3be-fe4b-4d96-b704-3f346c0aec17" path="/var/lib/kubelet/pods/a1a8e3be-fe4b-4d96-b704-3f346c0aec17/volumes"
Jul 24 03:24:45.346061 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.345906    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f5bc97f9d-wpfck"]
Jul 24 03:24:45.346541 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.346331    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1a8e3be-fe4b-4d96-b704-3f346c0aec17" containerName="console"
Jul 24 03:24:45.346541 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.346347    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1a8e3be-fe4b-4d96-b704-3f346c0aec17" containerName="console"
Jul 24 03:24:45.346541 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.346387    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1a8e3be-fe4b-4d96-b704-3f346c0aec17" containerName="console"
Jul 24 03:24:45.349171 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.349152    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.360234 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.360206    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5bc97f9d-wpfck"]
Jul 24 03:24:45.456110 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.456077    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-service-ca\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.456301 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.456122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-console-config\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.456301 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.456146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-serving-cert\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.456301 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.456211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-trusted-ca-bundle\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.456301 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.456290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-oauth-config\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.456442 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.456309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzkht\" (UniqueName: \"kubernetes.io/projected/9a77a591-df33-4d69-964e-c2e8d390e8df-kube-api-access-xzkht\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.456442 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.456332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-oauth-serving-cert\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.557411 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.557371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-service-ca\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.557411 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.557417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-console-config\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.557684 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.557438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-serving-cert\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.557684 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.557454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-trusted-ca-bundle\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.557684 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.557595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-oauth-config\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.557684 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.557636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzkht\" (UniqueName: \"kubernetes.io/projected/9a77a591-df33-4d69-964e-c2e8d390e8df-kube-api-access-xzkht\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.557684 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.557676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-oauth-serving-cert\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.558249 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.558220    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-console-config\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.558630 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.558323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-service-ca\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.558630 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.558361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-trusted-ca-bundle\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.558834 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.558379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-oauth-serving-cert\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.560141 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.560119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-serving-cert\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.560141 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.560138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-oauth-config\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.566428 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.566404    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzkht\" (UniqueName: \"kubernetes.io/projected/9a77a591-df33-4d69-964e-c2e8d390e8df-kube-api-access-xzkht\") pod \"console-6f5bc97f9d-wpfck\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") " pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.658441 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.658358    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:45.811235 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:45.811204    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5bc97f9d-wpfck"]
Jul 24 03:24:45.814819 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:24:45.814791    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a77a591_df33_4d69_964e_c2e8d390e8df.slice/crio-bed1850b7e8ecaed2bbf865d0b43098847692c2f23f6d5a9255c4e40018afb16 WatchSource:0}: Error finding container bed1850b7e8ecaed2bbf865d0b43098847692c2f23f6d5a9255c4e40018afb16: Status 404 returned error can't find the container with id bed1850b7e8ecaed2bbf865d0b43098847692c2f23f6d5a9255c4e40018afb16
Jul 24 03:24:46.214060 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:46.214020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5bc97f9d-wpfck" event={"ID":"9a77a591-df33-4d69-964e-c2e8d390e8df","Type":"ContainerStarted","Data":"ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f"}
Jul 24 03:24:46.214301 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:46.214069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5bc97f9d-wpfck" event={"ID":"9a77a591-df33-4d69-964e-c2e8d390e8df","Type":"ContainerStarted","Data":"bed1850b7e8ecaed2bbf865d0b43098847692c2f23f6d5a9255c4e40018afb16"}
Jul 24 03:24:46.249872 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:46.249817    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f5bc97f9d-wpfck" podStartSLOduration=1.249799831 podStartE2EDuration="1.249799831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:24:46.248106222 +0000 UTC m=+453.009014702" watchObservedRunningTime="2026-07-24 03:24:46.249799831 +0000 UTC m=+453.010708311"
Jul 24 03:24:55.659179 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:55.659129    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:55.659719 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:55.659207    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:55.663887 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:55.663868    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:56.257953 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:56.257924    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:24:56.300213 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:24:56.300178    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78dfb98c4c-82lct"]
Jul 24 03:25:07.398427 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.398392    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"]
Jul 24 03:25:07.401735 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.401718    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.404324 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.404302    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 03:25:07.404415 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.404305    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2tjcw\""
Jul 24 03:25:07.405354 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.405337    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 03:25:07.411135 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.411111    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"]
Jul 24 03:25:07.533487 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.533451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.533679 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.533507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hm88\" (UniqueName: \"kubernetes.io/projected/f26017ba-7b10-402d-bde3-8d3501fb795c-kube-api-access-5hm88\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.533679 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.533559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.634743 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.634704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.634905 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.634764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.634905 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.634801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hm88\" (UniqueName: \"kubernetes.io/projected/f26017ba-7b10-402d-bde3-8d3501fb795c-kube-api-access-5hm88\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.635103 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.635082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.635143 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.635114    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.643385 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.643356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hm88\" (UniqueName: \"kubernetes.io/projected/f26017ba-7b10-402d-bde3-8d3501fb795c-kube-api-access-5hm88\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.711387 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.711358    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:07.840841 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:07.840810    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"]
Jul 24 03:25:07.843752 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:25:07.843726    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26017ba_7b10_402d_bde3_8d3501fb795c.slice/crio-ce7cf208b5931e8bf22302e1df7019f720d5083a10a1444c26f3add85b3a9ce2 WatchSource:0}: Error finding container ce7cf208b5931e8bf22302e1df7019f720d5083a10a1444c26f3add85b3a9ce2: Status 404 returned error can't find the container with id ce7cf208b5931e8bf22302e1df7019f720d5083a10a1444c26f3add85b3a9ce2
Jul 24 03:25:08.300502 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:08.300471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c" event={"ID":"f26017ba-7b10-402d-bde3-8d3501fb795c","Type":"ContainerStarted","Data":"ce7cf208b5931e8bf22302e1df7019f720d5083a10a1444c26f3add85b3a9ce2"}
Jul 24 03:25:14.328854 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:14.328811    2577 generic.go:356] "Generic (PLEG): container finished" podID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerID="fab2a1452939fb5fda0e4bcb3982583e3bb6ce1debeb940bff4057c0b48ac941" exitCode=0
Jul 24 03:25:14.329316 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:14.328898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c" event={"ID":"f26017ba-7b10-402d-bde3-8d3501fb795c","Type":"ContainerDied","Data":"fab2a1452939fb5fda0e4bcb3982583e3bb6ce1debeb940bff4057c0b48ac941"}
Jul 24 03:25:17.342151 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:17.342113    2577 generic.go:356] "Generic (PLEG): container finished" podID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerID="3b5a4d86e9a822596aef5614c4615d3127e0f71fdc24b316c330dc6cf8d1004d" exitCode=0
Jul 24 03:25:17.342558 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:17.342194    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c" event={"ID":"f26017ba-7b10-402d-bde3-8d3501fb795c","Type":"ContainerDied","Data":"3b5a4d86e9a822596aef5614c4615d3127e0f71fdc24b316c330dc6cf8d1004d"}
Jul 24 03:25:21.319692 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:21.319644    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78dfb98c4c-82lct" podUID="b22a9df3-317a-48c7-ad80-accb5c59b4bf" containerName="console" containerID="cri-o://fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382" gracePeriod=15
Jul 24 03:25:23.812998 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.812969    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:25:23.885085 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.885052    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6kqb\" (UniqueName: \"kubernetes.io/projected/b22a9df3-317a-48c7-ad80-accb5c59b4bf-kube-api-access-m6kqb\") pod \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") "
Jul 24 03:25:23.885257 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.885093    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-trusted-ca-bundle\") pod \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") "
Jul 24 03:25:23.885257 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.885131    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-serving-cert\") pod \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") "
Jul 24 03:25:23.885257 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.885151    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-service-ca\") pod \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") "
Jul 24 03:25:23.885257 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.885183    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-oauth-config\") pod \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") "
Jul 24 03:25:23.885257 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.885220    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-oauth-serving-cert\") pod \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") "
Jul 24 03:25:23.885589 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.885559    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-service-ca" (OuterVolumeSpecName: "service-ca") pod "b22a9df3-317a-48c7-ad80-accb5c59b4bf" (UID: "b22a9df3-317a-48c7-ad80-accb5c59b4bf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:25:23.885683 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.885615    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b22a9df3-317a-48c7-ad80-accb5c59b4bf" (UID: "b22a9df3-317a-48c7-ad80-accb5c59b4bf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:25:23.885794 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.885695    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b22a9df3-317a-48c7-ad80-accb5c59b4bf" (UID: "b22a9df3-317a-48c7-ad80-accb5c59b4bf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:25:23.887546 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.887520    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b22a9df3-317a-48c7-ad80-accb5c59b4bf-kube-api-access-m6kqb" (OuterVolumeSpecName: "kube-api-access-m6kqb") pod "b22a9df3-317a-48c7-ad80-accb5c59b4bf" (UID: "b22a9df3-317a-48c7-ad80-accb5c59b4bf"). InnerVolumeSpecName "kube-api-access-m6kqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:25:23.887633 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.887545    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b22a9df3-317a-48c7-ad80-accb5c59b4bf" (UID: "b22a9df3-317a-48c7-ad80-accb5c59b4bf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:25:23.887633 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.887622    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b22a9df3-317a-48c7-ad80-accb5c59b4bf" (UID: "b22a9df3-317a-48c7-ad80-accb5c59b4bf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:25:23.986074 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.985973    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-config\") pod \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\" (UID: \"b22a9df3-317a-48c7-ad80-accb5c59b4bf\") "
Jul 24 03:25:23.986231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.986134    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-trusted-ca-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:23.986231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.986144    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-serving-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:23.986231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.986153    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:23.986231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.986174    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-oauth-config\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:23.986231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.986183    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-oauth-serving-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:23.986231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.986191    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6kqb\" (UniqueName: \"kubernetes.io/projected/b22a9df3-317a-48c7-ad80-accb5c59b4bf-kube-api-access-m6kqb\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:23.986545 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:23.986431    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-config" (OuterVolumeSpecName: "console-config") pod "b22a9df3-317a-48c7-ad80-accb5c59b4bf" (UID: "b22a9df3-317a-48c7-ad80-accb5c59b4bf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:25:24.087086 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.087043    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b22a9df3-317a-48c7-ad80-accb5c59b4bf-console-config\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:24.370026 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.369935    2577 generic.go:356] "Generic (PLEG): container finished" podID="b22a9df3-317a-48c7-ad80-accb5c59b4bf" containerID="fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382" exitCode=0
Jul 24 03:25:24.370026 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.369982    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dfb98c4c-82lct"
Jul 24 03:25:24.370284 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.370025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dfb98c4c-82lct" event={"ID":"b22a9df3-317a-48c7-ad80-accb5c59b4bf","Type":"ContainerDied","Data":"fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382"}
Jul 24 03:25:24.370284 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.370078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dfb98c4c-82lct" event={"ID":"b22a9df3-317a-48c7-ad80-accb5c59b4bf","Type":"ContainerDied","Data":"af668e5fb7dc5dde3bd71b23ba3367c991bf2704ca234b8a320e95db2dfe7880"}
Jul 24 03:25:24.370284 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.370104    2577 scope.go:117] "RemoveContainer" containerID="fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382"
Jul 24 03:25:24.372325 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.372290    2577 generic.go:356] "Generic (PLEG): container finished" podID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerID="f2a81eab77ecf5a40d9ed8d4792d6aea361df6900e8a7cf4b245d17c94922a6c" exitCode=0
Jul 24 03:25:24.372423 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.372362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c" event={"ID":"f26017ba-7b10-402d-bde3-8d3501fb795c","Type":"ContainerDied","Data":"f2a81eab77ecf5a40d9ed8d4792d6aea361df6900e8a7cf4b245d17c94922a6c"}
Jul 24 03:25:24.378950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.378934    2577 scope.go:117] "RemoveContainer" containerID="fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382"
Jul 24 03:25:24.379193 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:25:24.379175    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382\": container with ID starting with fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382 not found: ID does not exist" containerID="fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382"
Jul 24 03:25:24.379239 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.379202    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382"} err="failed to get container status \"fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382\": rpc error: code = NotFound desc = could not find container \"fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382\": container with ID starting with fe0c617f3489c4d256167d810f715d15223d9cbcf7be6bb5860667c129e9f382 not found: ID does not exist"
Jul 24 03:25:24.404621 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.404592    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78dfb98c4c-82lct"]
Jul 24 03:25:24.408402 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:24.408381    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78dfb98c4c-82lct"]
Jul 24 03:25:25.503496 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.503469    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:25.595903 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.595867    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-bundle\") pod \"f26017ba-7b10-402d-bde3-8d3501fb795c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") "
Jul 24 03:25:25.595903 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.595905    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hm88\" (UniqueName: \"kubernetes.io/projected/f26017ba-7b10-402d-bde3-8d3501fb795c-kube-api-access-5hm88\") pod \"f26017ba-7b10-402d-bde3-8d3501fb795c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") "
Jul 24 03:25:25.596126 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.595920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-util\") pod \"f26017ba-7b10-402d-bde3-8d3501fb795c\" (UID: \"f26017ba-7b10-402d-bde3-8d3501fb795c\") "
Jul 24 03:25:25.596570 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.596548    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-bundle" (OuterVolumeSpecName: "bundle") pod "f26017ba-7b10-402d-bde3-8d3501fb795c" (UID: "f26017ba-7b10-402d-bde3-8d3501fb795c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:25:25.598230 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.598207    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f26017ba-7b10-402d-bde3-8d3501fb795c-kube-api-access-5hm88" (OuterVolumeSpecName: "kube-api-access-5hm88") pod "f26017ba-7b10-402d-bde3-8d3501fb795c" (UID: "f26017ba-7b10-402d-bde3-8d3501fb795c"). InnerVolumeSpecName "kube-api-access-5hm88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:25:25.600111 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.600090    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-util" (OuterVolumeSpecName: "util") pod "f26017ba-7b10-402d-bde3-8d3501fb795c" (UID: "f26017ba-7b10-402d-bde3-8d3501fb795c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:25:25.640925 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.640862    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b22a9df3-317a-48c7-ad80-accb5c59b4bf" path="/var/lib/kubelet/pods/b22a9df3-317a-48c7-ad80-accb5c59b4bf/volumes"
Jul 24 03:25:25.696737 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.696704    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:25.696737 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.696731    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hm88\" (UniqueName: \"kubernetes.io/projected/f26017ba-7b10-402d-bde3-8d3501fb795c-kube-api-access-5hm88\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:25.696935 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:25.696747    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f26017ba-7b10-402d-bde3-8d3501fb795c-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:26.389951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:26.389904    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c" event={"ID":"f26017ba-7b10-402d-bde3-8d3501fb795c","Type":"ContainerDied","Data":"ce7cf208b5931e8bf22302e1df7019f720d5083a10a1444c26f3add85b3a9ce2"}
Jul 24 03:25:26.389951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:26.389949    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405l64c"
Jul 24 03:25:26.390150 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:26.389988    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce7cf208b5931e8bf22302e1df7019f720d5083a10a1444c26f3add85b3a9ce2"
Jul 24 03:25:30.386969 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.386929    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"]
Jul 24 03:25:30.387480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387338    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b22a9df3-317a-48c7-ad80-accb5c59b4bf" containerName="console"
Jul 24 03:25:30.387480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387357    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b22a9df3-317a-48c7-ad80-accb5c59b4bf" containerName="console"
Jul 24 03:25:30.387480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387386    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerName="pull"
Jul 24 03:25:30.387480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387395    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerName="pull"
Jul 24 03:25:30.387480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387410    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerName="util"
Jul 24 03:25:30.387480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387417    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerName="util"
Jul 24 03:25:30.387480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387426    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerName="extract"
Jul 24 03:25:30.387480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387433    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerName="extract"
Jul 24 03:25:30.387850 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387500    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b22a9df3-317a-48c7-ad80-accb5c59b4bf" containerName="console"
Jul 24 03:25:30.387850 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.387514    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f26017ba-7b10-402d-bde3-8d3501fb795c" containerName="extract"
Jul 24 03:25:30.392328 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.392307    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"
Jul 24 03:25:30.394966 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.394947    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:25:30.395102 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.395052    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 24 03:25:30.395260 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.395244    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-2jfks\""
Jul 24 03:25:30.402899 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.402876    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"]
Jul 24 03:25:30.430709 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.430680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5dedcaa-7565-42b0-87f3-47e07e48f880-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mpm98\" (UID: \"c5dedcaa-7565-42b0-87f3-47e07e48f880\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"
Jul 24 03:25:30.430887 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.430740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdrj5\" (UniqueName: \"kubernetes.io/projected/c5dedcaa-7565-42b0-87f3-47e07e48f880-kube-api-access-sdrj5\") pod \"cert-manager-operator-controller-manager-55c444bf86-mpm98\" (UID: \"c5dedcaa-7565-42b0-87f3-47e07e48f880\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"
Jul 24 03:25:30.531912 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.531860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdrj5\" (UniqueName: \"kubernetes.io/projected/c5dedcaa-7565-42b0-87f3-47e07e48f880-kube-api-access-sdrj5\") pod \"cert-manager-operator-controller-manager-55c444bf86-mpm98\" (UID: \"c5dedcaa-7565-42b0-87f3-47e07e48f880\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"
Jul 24 03:25:30.532119 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.531932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5dedcaa-7565-42b0-87f3-47e07e48f880-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mpm98\" (UID: \"c5dedcaa-7565-42b0-87f3-47e07e48f880\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"
Jul 24 03:25:30.532423 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.532404    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5dedcaa-7565-42b0-87f3-47e07e48f880-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mpm98\" (UID: \"c5dedcaa-7565-42b0-87f3-47e07e48f880\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"
Jul 24 03:25:30.542594 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.542575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdrj5\" (UniqueName: \"kubernetes.io/projected/c5dedcaa-7565-42b0-87f3-47e07e48f880-kube-api-access-sdrj5\") pod \"cert-manager-operator-controller-manager-55c444bf86-mpm98\" (UID: \"c5dedcaa-7565-42b0-87f3-47e07e48f880\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"
Jul 24 03:25:30.701031 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.700992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"
Jul 24 03:25:30.834213 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:30.834181    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98"]
Jul 24 03:25:30.837691 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:25:30.837661    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5dedcaa_7565_42b0_87f3_47e07e48f880.slice/crio-7ecf8ae2eda7926908900fabe855ed0db9ea8e82f9dcdc42e64f2f1edaf8554d WatchSource:0}: Error finding container 7ecf8ae2eda7926908900fabe855ed0db9ea8e82f9dcdc42e64f2f1edaf8554d: Status 404 returned error can't find the container with id 7ecf8ae2eda7926908900fabe855ed0db9ea8e82f9dcdc42e64f2f1edaf8554d
Jul 24 03:25:31.410693 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:31.410658    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98" event={"ID":"c5dedcaa-7565-42b0-87f3-47e07e48f880","Type":"ContainerStarted","Data":"7ecf8ae2eda7926908900fabe855ed0db9ea8e82f9dcdc42e64f2f1edaf8554d"}
Jul 24 03:25:33.420224 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:33.420185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98" event={"ID":"c5dedcaa-7565-42b0-87f3-47e07e48f880","Type":"ContainerStarted","Data":"14309a6d4696de71409f56ab73388c2f8ce647f198a8b05e1c1eaeda3ad7fdab"}
Jul 24 03:25:33.452087 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:33.452037    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mpm98" podStartSLOduration=1.883994122 podStartE2EDuration="3.452021286s" totalImagesPullingTime="1.568027164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:25:33.449944123 +0000 UTC m=+500.210852604" watchObservedRunningTime="2026-07-24 03:25:33.452021286 +0000 UTC m=+500.212929765"
Jul 24 03:25:34.196916 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.196874    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"]
Jul 24 03:25:34.200356 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.200332    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.215177 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.215155    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 03:25:34.215324 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.215231    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2tjcw\""
Jul 24 03:25:34.226013 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.225986    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 03:25:34.228104 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.228081    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"]
Jul 24 03:25:34.262740 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.262712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.262907 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.262765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.262907 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.262844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvksx\" (UniqueName: \"kubernetes.io/projected/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-kube-api-access-mvksx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.364240 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.364205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.364240 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.364248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.364492 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.364307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvksx\" (UniqueName: \"kubernetes.io/projected/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-kube-api-access-mvksx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.364617 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.364598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.364686 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.364668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.372951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.372921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvksx\" (UniqueName: \"kubernetes.io/projected/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-kube-api-access-mvksx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.509862 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.509772    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:34.647462 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:34.647402    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"]
Jul 24 03:25:34.649897 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:25:34.649862    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f997d3b_86e6_4b0f_96d7_cc0834f6dc35.slice/crio-c638c0100ce6f3ab9a33c5548c84ee58044d2c37a6fa00a4e175d4b69bffd288 WatchSource:0}: Error finding container c638c0100ce6f3ab9a33c5548c84ee58044d2c37a6fa00a4e175d4b69bffd288: Status 404 returned error can't find the container with id c638c0100ce6f3ab9a33c5548c84ee58044d2c37a6fa00a4e175d4b69bffd288
Jul 24 03:25:35.396006 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.395972    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cf9tk"]
Jul 24 03:25:35.399312 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.399292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:35.401893 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.401873    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 03:25:35.402960 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.402944    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-85lpm\""
Jul 24 03:25:35.403014 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.402988    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 03:25:35.407775 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.407725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cf9tk"]
Jul 24 03:25:35.430716 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.430682    2577 generic.go:356] "Generic (PLEG): container finished" podID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerID="bd5cdb039d8b925e41ddd26318632421e09f8b89e06b75b815cf64b7929ce6c1" exitCode=0
Jul 24 03:25:35.430836 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.430738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz" event={"ID":"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35","Type":"ContainerDied","Data":"bd5cdb039d8b925e41ddd26318632421e09f8b89e06b75b815cf64b7929ce6c1"}
Jul 24 03:25:35.430836 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.430782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz" event={"ID":"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35","Type":"ContainerStarted","Data":"c638c0100ce6f3ab9a33c5548c84ee58044d2c37a6fa00a4e175d4b69bffd288"}
Jul 24 03:25:35.474100 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.474067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93c0b930-dbe1-402c-b4f0-42513dc837d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cf9tk\" (UID: \"93c0b930-dbe1-402c-b4f0-42513dc837d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:35.474261 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.474110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcwtw\" (UniqueName: \"kubernetes.io/projected/93c0b930-dbe1-402c-b4f0-42513dc837d0-kube-api-access-mcwtw\") pod \"cert-manager-webhook-6486ffd997-cf9tk\" (UID: \"93c0b930-dbe1-402c-b4f0-42513dc837d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:35.575206 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.575167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93c0b930-dbe1-402c-b4f0-42513dc837d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cf9tk\" (UID: \"93c0b930-dbe1-402c-b4f0-42513dc837d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:35.575597 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.575239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcwtw\" (UniqueName: \"kubernetes.io/projected/93c0b930-dbe1-402c-b4f0-42513dc837d0-kube-api-access-mcwtw\") pod \"cert-manager-webhook-6486ffd997-cf9tk\" (UID: \"93c0b930-dbe1-402c-b4f0-42513dc837d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:35.583841 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.583811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93c0b930-dbe1-402c-b4f0-42513dc837d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cf9tk\" (UID: \"93c0b930-dbe1-402c-b4f0-42513dc837d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:35.583934 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.583874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcwtw\" (UniqueName: \"kubernetes.io/projected/93c0b930-dbe1-402c-b4f0-42513dc837d0-kube-api-access-mcwtw\") pod \"cert-manager-webhook-6486ffd997-cf9tk\" (UID: \"93c0b930-dbe1-402c-b4f0-42513dc837d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:35.709096 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.709063    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:35.850623 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:35.850600    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cf9tk"]
Jul 24 03:25:35.853050 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:25:35.853012    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93c0b930_dbe1_402c_b4f0_42513dc837d0.slice/crio-4c417d8aab09290e61a329718b336d4354881a80eaf18225bd39894c36b148a7 WatchSource:0}: Error finding container 4c417d8aab09290e61a329718b336d4354881a80eaf18225bd39894c36b148a7: Status 404 returned error can't find the container with id 4c417d8aab09290e61a329718b336d4354881a80eaf18225bd39894c36b148a7
Jul 24 03:25:36.438196 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:36.438136    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk" event={"ID":"93c0b930-dbe1-402c-b4f0-42513dc837d0","Type":"ContainerStarted","Data":"4c417d8aab09290e61a329718b336d4354881a80eaf18225bd39894c36b148a7"}
Jul 24 03:25:37.025001 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.024931    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"]
Jul 24 03:25:37.034010 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.033971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"
Jul 24 03:25:37.034856 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.034826    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"]
Jul 24 03:25:37.037011 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.036988    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-tr95f\""
Jul 24 03:25:37.088137 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.088097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9spfb\" (UniqueName: \"kubernetes.io/projected/d262a161-c546-49c4-8311-0b3c4fcf5e2b-kube-api-access-9spfb\") pod \"cert-manager-cainjector-559cdd8c87-xb6qb\" (UID: \"d262a161-c546-49c4-8311-0b3c4fcf5e2b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"
Jul 24 03:25:37.088350 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.088165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d262a161-c546-49c4-8311-0b3c4fcf5e2b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xb6qb\" (UID: \"d262a161-c546-49c4-8311-0b3c4fcf5e2b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"
Jul 24 03:25:37.189305 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.189091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9spfb\" (UniqueName: \"kubernetes.io/projected/d262a161-c546-49c4-8311-0b3c4fcf5e2b-kube-api-access-9spfb\") pod \"cert-manager-cainjector-559cdd8c87-xb6qb\" (UID: \"d262a161-c546-49c4-8311-0b3c4fcf5e2b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"
Jul 24 03:25:37.189305 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.189153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d262a161-c546-49c4-8311-0b3c4fcf5e2b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xb6qb\" (UID: \"d262a161-c546-49c4-8311-0b3c4fcf5e2b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"
Jul 24 03:25:37.197978 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.197942    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d262a161-c546-49c4-8311-0b3c4fcf5e2b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xb6qb\" (UID: \"d262a161-c546-49c4-8311-0b3c4fcf5e2b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"
Jul 24 03:25:37.198437 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.198363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9spfb\" (UniqueName: \"kubernetes.io/projected/d262a161-c546-49c4-8311-0b3c4fcf5e2b-kube-api-access-9spfb\") pod \"cert-manager-cainjector-559cdd8c87-xb6qb\" (UID: \"d262a161-c546-49c4-8311-0b3c4fcf5e2b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"
Jul 24 03:25:37.348309 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:37.348200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"
Jul 24 03:25:38.697967 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:38.697674    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb"]
Jul 24 03:25:38.699791 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:25:38.699748    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd262a161_c546_49c4_8311_0b3c4fcf5e2b.slice/crio-5adcbcfec48e06b4bdc3599514425bd21c1298863baddd3f118cd6ada2bd8936 WatchSource:0}: Error finding container 5adcbcfec48e06b4bdc3599514425bd21c1298863baddd3f118cd6ada2bd8936: Status 404 returned error can't find the container with id 5adcbcfec48e06b4bdc3599514425bd21c1298863baddd3f118cd6ada2bd8936
Jul 24 03:25:39.455952 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:39.455916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk" event={"ID":"93c0b930-dbe1-402c-b4f0-42513dc837d0","Type":"ContainerStarted","Data":"f61b0efa5143131646bfb9b7806d2769ea6dbfa2b31087659d616c3512818d89"}
Jul 24 03:25:39.456188 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:39.455971    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:39.457511 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:39.457481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb" event={"ID":"d262a161-c546-49c4-8311-0b3c4fcf5e2b","Type":"ContainerStarted","Data":"0690e8159b4e65ea5c6ab1625f1bf612d52e80d8721ec079b8d65e84fde80548"}
Jul 24 03:25:39.457615 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:39.457523    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb" event={"ID":"d262a161-c546-49c4-8311-0b3c4fcf5e2b","Type":"ContainerStarted","Data":"5adcbcfec48e06b4bdc3599514425bd21c1298863baddd3f118cd6ada2bd8936"}
Jul 24 03:25:39.459125 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:39.459102    2577 generic.go:356] "Generic (PLEG): container finished" podID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerID="6d7bff5ea8d41f3cc6ce3a08e1713820b22aaeb6cbffab45bc53aaebaf929bc6" exitCode=0
Jul 24 03:25:39.459226 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:39.459154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz" event={"ID":"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35","Type":"ContainerDied","Data":"6d7bff5ea8d41f3cc6ce3a08e1713820b22aaeb6cbffab45bc53aaebaf929bc6"}
Jul 24 03:25:39.473627 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:39.473578    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk" podStartSLOduration=1.753706616 podStartE2EDuration="4.473562603s" totalImagesPullingTime="2.719855987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:25:39.471956597 +0000 UTC m=+506.232865079" watchObservedRunningTime="2026-07-24 03:25:39.473562603 +0000 UTC m=+506.234471084"
Jul 24 03:25:39.504064 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:39.504018    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xb6qb" podStartSLOduration=2.504001702 podStartE2EDuration="2.504001702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:25:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:25:39.502344775 +0000 UTC m=+506.263253255" watchObservedRunningTime="2026-07-24 03:25:39.504001702 +0000 UTC m=+506.264910181"
Jul 24 03:25:40.465413 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:40.465375    2577 generic.go:356] "Generic (PLEG): container finished" podID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerID="1970d9f57a34361a0cac209f5619cb548320bb5ffb6414b0ba53b5d39cba975a" exitCode=0
Jul 24 03:25:40.465808 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:40.465464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz" event={"ID":"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35","Type":"ContainerDied","Data":"1970d9f57a34361a0cac209f5619cb548320bb5ffb6414b0ba53b5d39cba975a"}
Jul 24 03:25:41.597012 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.596985    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:41.628501 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.628471    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-bundle\") pod \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") "
Jul 24 03:25:41.628501 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.628501    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvksx\" (UniqueName: \"kubernetes.io/projected/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-kube-api-access-mvksx\") pod \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") "
Jul 24 03:25:41.628684 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.628546    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-util\") pod \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\" (UID: \"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35\") "
Jul 24 03:25:41.628898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.628867    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-bundle" (OuterVolumeSpecName: "bundle") pod "4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" (UID: "4f997d3b-86e6-4b0f-96d7-cc0834f6dc35"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:25:41.630772 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.630748    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-kube-api-access-mvksx" (OuterVolumeSpecName: "kube-api-access-mvksx") pod "4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" (UID: "4f997d3b-86e6-4b0f-96d7-cc0834f6dc35"). InnerVolumeSpecName "kube-api-access-mvksx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:25:41.633824 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.633797    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-util" (OuterVolumeSpecName: "util") pod "4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" (UID: "4f997d3b-86e6-4b0f-96d7-cc0834f6dc35"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:25:41.729220 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.729127    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:41.729220 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.729163    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvksx\" (UniqueName: \"kubernetes.io/projected/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-kube-api-access-mvksx\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:41.729220 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:41.729172    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f997d3b-86e6-4b0f-96d7-cc0834f6dc35-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:25:42.474977 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:42.474942    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz" event={"ID":"4f997d3b-86e6-4b0f-96d7-cc0834f6dc35","Type":"ContainerDied","Data":"c638c0100ce6f3ab9a33c5548c84ee58044d2c37a6fa00a4e175d4b69bffd288"}
Jul 24 03:25:42.475220 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:42.475019    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c638c0100ce6f3ab9a33c5548c84ee58044d2c37a6fa00a4e175d4b69bffd288"
Jul 24 03:25:42.475220 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:42.474955    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f289wz"
Jul 24 03:25:45.221196 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.221160    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d77s4"]
Jul 24 03:25:45.221604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.221506    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerName="pull"
Jul 24 03:25:45.221604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.221520    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerName="pull"
Jul 24 03:25:45.221604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.221534    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerName="util"
Jul 24 03:25:45.221604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.221543    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerName="util"
Jul 24 03:25:45.221604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.221558    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerName="extract"
Jul 24 03:25:45.221604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.221563    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerName="extract"
Jul 24 03:25:45.221786 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.221633    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f997d3b-86e6-4b0f-96d7-cc0834f6dc35" containerName="extract"
Jul 24 03:25:45.224777 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.224760    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-d77s4"
Jul 24 03:25:45.227121 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.227095    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9st6l\""
Jul 24 03:25:45.233146 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.233118    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d77s4"]
Jul 24 03:25:45.357595 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.357555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd-bound-sa-token\") pod \"cert-manager-55b4d954d5-d77s4\" (UID: \"3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd\") " pod="cert-manager/cert-manager-55b4d954d5-d77s4"
Jul 24 03:25:45.357595 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.357596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lj2c\" (UniqueName: \"kubernetes.io/projected/3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd-kube-api-access-7lj2c\") pod \"cert-manager-55b4d954d5-d77s4\" (UID: \"3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd\") " pod="cert-manager/cert-manager-55b4d954d5-d77s4"
Jul 24 03:25:45.459025 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.458989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd-bound-sa-token\") pod \"cert-manager-55b4d954d5-d77s4\" (UID: \"3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd\") " pod="cert-manager/cert-manager-55b4d954d5-d77s4"
Jul 24 03:25:45.459025 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.459030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lj2c\" (UniqueName: \"kubernetes.io/projected/3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd-kube-api-access-7lj2c\") pod \"cert-manager-55b4d954d5-d77s4\" (UID: \"3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd\") " pod="cert-manager/cert-manager-55b4d954d5-d77s4"
Jul 24 03:25:45.467839 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.467810    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cf9tk"
Jul 24 03:25:45.467979 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.467867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd-bound-sa-token\") pod \"cert-manager-55b4d954d5-d77s4\" (UID: \"3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd\") " pod="cert-manager/cert-manager-55b4d954d5-d77s4"
Jul 24 03:25:45.467979 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.467913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lj2c\" (UniqueName: \"kubernetes.io/projected/3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd-kube-api-access-7lj2c\") pod \"cert-manager-55b4d954d5-d77s4\" (UID: \"3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd\") " pod="cert-manager/cert-manager-55b4d954d5-d77s4"
Jul 24 03:25:45.534233 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.534155    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-d77s4"
Jul 24 03:25:45.690811 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:45.690783    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d77s4"]
Jul 24 03:25:45.693053 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:25:45.693024    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ed7e49a_02d8_4c7e_bfa3_73510fa80ddd.slice/crio-c69ec262add1a0c2725a8987365950e0718677e74c3c9efdbcbc08f2f62174ef WatchSource:0}: Error finding container c69ec262add1a0c2725a8987365950e0718677e74c3c9efdbcbc08f2f62174ef: Status 404 returned error can't find the container with id c69ec262add1a0c2725a8987365950e0718677e74c3c9efdbcbc08f2f62174ef
Jul 24 03:25:46.493365 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:46.493325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-d77s4" event={"ID":"3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd","Type":"ContainerStarted","Data":"b7ef35227a137f7fc628054f31245248a5d63f0e9b6ab7e598cb4c54c1ab9d9b"}
Jul 24 03:25:46.493365 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:46.493367    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-d77s4" event={"ID":"3ed7e49a-02d8-4c7e-bfa3-73510fa80ddd","Type":"ContainerStarted","Data":"c69ec262add1a0c2725a8987365950e0718677e74c3c9efdbcbc08f2f62174ef"}
Jul 24 03:25:46.510109 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:46.510059    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-d77s4" podStartSLOduration=1.510043793 podStartE2EDuration="1.510043793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:25:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:25:46.508802943 +0000 UTC m=+513.269711403" watchObservedRunningTime="2026-07-24 03:25:46.510043793 +0000 UTC m=+513.270952272"
Jul 24 03:25:57.253063 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.253030    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"]
Jul 24 03:25:57.260444 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.260425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.263279 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.263245    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 24 03:25:57.264130 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.264112    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 24 03:25:57.264225 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.264206    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2tjcw\""
Jul 24 03:25:57.267553 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.267526    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"]
Jul 24 03:25:57.355396 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.355358    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.355572 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.355411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmlfm\" (UniqueName: \"kubernetes.io/projected/f64b422b-ed1e-4483-9341-dcd707145828-kube-api-access-gmlfm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.355572 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.355507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.456868 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.456835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.457044 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.456895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.457044 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.456917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmlfm\" (UniqueName: \"kubernetes.io/projected/f64b422b-ed1e-4483-9341-dcd707145828-kube-api-access-gmlfm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.457251 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.457231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.457325 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.457289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.466118 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.466097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmlfm\" (UniqueName: \"kubernetes.io/projected/f64b422b-ed1e-4483-9341-dcd707145828-kube-api-access-gmlfm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.571772 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.571702    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:25:57.711904 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:57.711832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"]
Jul 24 03:25:57.714757 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:25:57.714730    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64b422b_ed1e_4483_9341_dcd707145828.slice/crio-7b40f16ae44223280a8b2e704bdb21e089f6c986630766cf36cd53cad86880b6 WatchSource:0}: Error finding container 7b40f16ae44223280a8b2e704bdb21e089f6c986630766cf36cd53cad86880b6: Status 404 returned error can't find the container with id 7b40f16ae44223280a8b2e704bdb21e089f6c986630766cf36cd53cad86880b6
Jul 24 03:25:58.543084 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:58.543050    2577 generic.go:356] "Generic (PLEG): container finished" podID="f64b422b-ed1e-4483-9341-dcd707145828" containerID="a6585f3c20cfd7c3fa2056825afb27f1bf27b43e1ee41921b4ad62080fae0a6b" exitCode=0
Jul 24 03:25:58.543473 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:58.543142    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x" event={"ID":"f64b422b-ed1e-4483-9341-dcd707145828","Type":"ContainerDied","Data":"a6585f3c20cfd7c3fa2056825afb27f1bf27b43e1ee41921b4ad62080fae0a6b"}
Jul 24 03:25:58.543473 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:58.543190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x" event={"ID":"f64b422b-ed1e-4483-9341-dcd707145828","Type":"ContainerStarted","Data":"7b40f16ae44223280a8b2e704bdb21e089f6c986630766cf36cd53cad86880b6"}
Jul 24 03:25:59.549587 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:59.549553    2577 generic.go:356] "Generic (PLEG): container finished" podID="f64b422b-ed1e-4483-9341-dcd707145828" containerID="ef91a50e83dba2d9fab4c7f537a0a3f390088cec5e1d0ce347b9b9fa41d9e64a" exitCode=0
Jul 24 03:25:59.549966 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:25:59.549644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x" event={"ID":"f64b422b-ed1e-4483-9341-dcd707145828","Type":"ContainerDied","Data":"ef91a50e83dba2d9fab4c7f537a0a3f390088cec5e1d0ce347b9b9fa41d9e64a"}
Jul 24 03:26:00.555960 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:00.555926    2577 generic.go:356] "Generic (PLEG): container finished" podID="f64b422b-ed1e-4483-9341-dcd707145828" containerID="0eeaf2a220daf5951b1f87aaff596a8c9ecb35e5e475c243c661064ecc4f4b0d" exitCode=0
Jul 24 03:26:00.555960 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:00.555965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x" event={"ID":"f64b422b-ed1e-4483-9341-dcd707145828","Type":"ContainerDied","Data":"0eeaf2a220daf5951b1f87aaff596a8c9ecb35e5e475c243c661064ecc4f4b0d"}
Jul 24 03:26:01.690405 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.690380    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:26:01.794027 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.793996    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmlfm\" (UniqueName: \"kubernetes.io/projected/f64b422b-ed1e-4483-9341-dcd707145828-kube-api-access-gmlfm\") pod \"f64b422b-ed1e-4483-9341-dcd707145828\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") "
Jul 24 03:26:01.794214 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.794091    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-bundle\") pod \"f64b422b-ed1e-4483-9341-dcd707145828\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") "
Jul 24 03:26:01.794214 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.794135    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-util\") pod \"f64b422b-ed1e-4483-9341-dcd707145828\" (UID: \"f64b422b-ed1e-4483-9341-dcd707145828\") "
Jul 24 03:26:01.794838 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.794813    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-bundle" (OuterVolumeSpecName: "bundle") pod "f64b422b-ed1e-4483-9341-dcd707145828" (UID: "f64b422b-ed1e-4483-9341-dcd707145828"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:26:01.796435 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.796399    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64b422b-ed1e-4483-9341-dcd707145828-kube-api-access-gmlfm" (OuterVolumeSpecName: "kube-api-access-gmlfm") pod "f64b422b-ed1e-4483-9341-dcd707145828" (UID: "f64b422b-ed1e-4483-9341-dcd707145828"). InnerVolumeSpecName "kube-api-access-gmlfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:26:01.800100 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.800075    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-util" (OuterVolumeSpecName: "util") pod "f64b422b-ed1e-4483-9341-dcd707145828" (UID: "f64b422b-ed1e-4483-9341-dcd707145828"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:26:01.895492 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.895405    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:26:01.895492 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.895432    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f64b422b-ed1e-4483-9341-dcd707145828-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:26:01.895492 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:01.895441    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmlfm\" (UniqueName: \"kubernetes.io/projected/f64b422b-ed1e-4483-9341-dcd707145828-kube-api-access-gmlfm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:26:02.566230 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:02.566198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x" event={"ID":"f64b422b-ed1e-4483-9341-dcd707145828","Type":"ContainerDied","Data":"7b40f16ae44223280a8b2e704bdb21e089f6c986630766cf36cd53cad86880b6"}
Jul 24 03:26:02.566426 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:02.566244    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwvv7x"
Jul 24 03:26:02.566426 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:02.566249    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b40f16ae44223280a8b2e704bdb21e089f6c986630766cf36cd53cad86880b6"
Jul 24 03:26:11.058242 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.058144    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"]
Jul 24 03:26:11.058646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.058452    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64b422b-ed1e-4483-9341-dcd707145828" containerName="extract"
Jul 24 03:26:11.058646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.058463    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64b422b-ed1e-4483-9341-dcd707145828" containerName="extract"
Jul 24 03:26:11.058646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.058473    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64b422b-ed1e-4483-9341-dcd707145828" containerName="util"
Jul 24 03:26:11.058646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.058479    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64b422b-ed1e-4483-9341-dcd707145828" containerName="util"
Jul 24 03:26:11.058646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.058488    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64b422b-ed1e-4483-9341-dcd707145828" containerName="pull"
Jul 24 03:26:11.058646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.058493    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64b422b-ed1e-4483-9341-dcd707145828" containerName="pull"
Jul 24 03:26:11.058646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.058548    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f64b422b-ed1e-4483-9341-dcd707145828" containerName="extract"
Jul 24 03:26:11.061504 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.061488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.070777 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:26:11.070742    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" is forbidden: User \"system:node:ip-10-0-132-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-49.ec2.internal' and this object" podUID="5065f5e1-78e0-4660-ade5-65c9788bd5c2" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.070777 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:26:11.070756    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-q7bbg\" is forbidden: User \"system:node:ip-10-0-132-49.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-49.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q7bbg\"" type="*v1.Secret"
Jul 24 03:26:11.070943 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:26:11.070836    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-132-49.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-49.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret"
Jul 24 03:26:11.070943 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:26:11.070908    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-132-49.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-49.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret"
Jul 24 03:26:11.070943 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:26:11.070916    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-49.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-49.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 24 03:26:11.071803 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.071785    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 03:26:11.118186 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.118151    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"]
Jul 24 03:26:11.169045 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.168994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.169229 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.169089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbtc6\" (UniqueName: \"kubernetes.io/projected/5065f5e1-78e0-4660-ade5-65c9788bd5c2-kube-api-access-bbtc6\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.169229 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.169128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5065f5e1-78e0-4660-ade5-65c9788bd5c2-manifests\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.169229 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.169149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5065f5e1-78e0-4660-ade5-65c9788bd5c2-tmp\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.169229 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.169215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.270511 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.270459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbtc6\" (UniqueName: \"kubernetes.io/projected/5065f5e1-78e0-4660-ade5-65c9788bd5c2-kube-api-access-bbtc6\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.270511 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.270515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5065f5e1-78e0-4660-ade5-65c9788bd5c2-manifests\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.270786 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.270544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5065f5e1-78e0-4660-ade5-65c9788bd5c2-tmp\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.270786 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.270594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.270786 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.270647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.270953 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.270932    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5065f5e1-78e0-4660-ade5-65c9788bd5c2-manifests\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:11.272965 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:11.272943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5065f5e1-78e0-4660-ade5-65c9788bd5c2-tmp\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:12.256033 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.255999    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 03:26:12.265647 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.265622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbtc6\" (UniqueName: \"kubernetes.io/projected/5065f5e1-78e0-4660-ade5-65c9788bd5c2-kube-api-access-bbtc6\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:12.271383 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:26:12.271355    2577 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition
Jul 24 03:26:12.271491 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:26:12.271445    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-apiservice-cert podName:5065f5e1-78e0-4660-ade5-65c9788bd5c2 nodeName:}" failed. No retries permitted until 2026-07-24 03:26:12.771426946 +0000 UTC m=+539.532335404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-apiservice-cert") pod "opendatahub-operator-controller-manager-67f4587fcc-tmkbp" (UID: "5065f5e1-78e0-4660-ade5-65c9788bd5c2") : failed to sync secret cache: timed out waiting for the condition
Jul 24 03:26:12.271491 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:26:12.271355    2577 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition
Jul 24 03:26:12.271631 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:26:12.271543    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-webhook-cert podName:5065f5e1-78e0-4660-ade5-65c9788bd5c2 nodeName:}" failed. No retries permitted until 2026-07-24 03:26:12.771522756 +0000 UTC m=+539.532431214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-webhook-cert") pod "opendatahub-operator-controller-manager-67f4587fcc-tmkbp" (UID: "5065f5e1-78e0-4660-ade5-65c9788bd5c2") : failed to sync secret cache: timed out waiting for the condition
Jul 24 03:26:12.407558 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.407522    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 03:26:12.466491 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.466455    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 03:26:12.527807 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.527729    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q7bbg\""
Jul 24 03:26:12.783469 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.783354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:12.783469 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.783447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:12.785907 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.785874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:12.786027 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.785970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5065f5e1-78e0-4660-ade5-65c9788bd5c2-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-tmkbp\" (UID: \"5065f5e1-78e0-4660-ade5-65c9788bd5c2\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:12.871684 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:12.871654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:13.013647 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:13.013625    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"]
Jul 24 03:26:13.015738 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:26:13.015707    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5065f5e1_78e0_4660_ade5_65c9788bd5c2.slice/crio-7b1cea218fa9969e2a1015fac78f0f83ebd2edf70b351534e82f0ac597cad840 WatchSource:0}: Error finding container 7b1cea218fa9969e2a1015fac78f0f83ebd2edf70b351534e82f0ac597cad840: Status 404 returned error can't find the container with id 7b1cea218fa9969e2a1015fac78f0f83ebd2edf70b351534e82f0ac597cad840
Jul 24 03:26:13.615230 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:13.615145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp" event={"ID":"5065f5e1-78e0-4660-ade5-65c9788bd5c2","Type":"ContainerStarted","Data":"7b1cea218fa9969e2a1015fac78f0f83ebd2edf70b351534e82f0ac597cad840"}
Jul 24 03:26:16.629913 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:16.629881    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp" event={"ID":"5065f5e1-78e0-4660-ade5-65c9788bd5c2","Type":"ContainerStarted","Data":"05694dddf3a9bd645b698176ce3664a05df742ca92b581b91d5b13928fcacb73"}
Jul 24 03:26:17.636360 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:17.636323    2577 generic.go:356] "Generic (PLEG): container finished" podID="5065f5e1-78e0-4660-ade5-65c9788bd5c2" containerID="05694dddf3a9bd645b698176ce3664a05df742ca92b581b91d5b13928fcacb73" exitCode=0
Jul 24 03:26:17.642070 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:17.642036    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp" event={"ID":"5065f5e1-78e0-4660-ade5-65c9788bd5c2","Type":"ContainerDied","Data":"05694dddf3a9bd645b698176ce3664a05df742ca92b581b91d5b13928fcacb73"}
Jul 24 03:26:18.643852 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:18.643819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp" event={"ID":"5065f5e1-78e0-4660-ade5-65c9788bd5c2","Type":"ContainerStarted","Data":"3056a9ff313594cd04ace7d4e1f787756732402e6f4a3db3b8f5a62758d16e7b"}
Jul 24 03:26:18.644238 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:18.643901    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:18.675229 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:18.675173    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp" podStartSLOduration=4.524520591 podStartE2EDuration="8.675155609s" totalImagesPullingTime="3.146526446s" totalInitContainerRuntime="1.004108572s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:26:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:26:18.674421387 +0000 UTC m=+545.435329869" watchObservedRunningTime="2026-07-24 03:26:18.675155609 +0000 UTC m=+545.436064088"
Jul 24 03:26:29.650533 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:29.650502    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-tmkbp"
Jul 24 03:26:32.651633 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.651594    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"]
Jul 24 03:26:32.665561 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.665530    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.670004 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.669979    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 03:26:32.670625 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.670601    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j4jv4\""
Jul 24 03:26:32.670745 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.670599    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 03:26:32.670745 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.670599    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 03:26:32.670745 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.670668    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 03:26:32.730531 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.730495    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"]
Jul 24 03:26:32.737067 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.737034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4d41a76-0f08-4a04-bbbe-a96749e38076-tmp\") pod \"kube-auth-proxy-5f459d4668-2qzl2\" (UID: \"c4d41a76-0f08-4a04-bbbe-a96749e38076\") " pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.737231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.737174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crzgp\" (UniqueName: \"kubernetes.io/projected/c4d41a76-0f08-4a04-bbbe-a96749e38076-kube-api-access-crzgp\") pod \"kube-auth-proxy-5f459d4668-2qzl2\" (UID: \"c4d41a76-0f08-4a04-bbbe-a96749e38076\") " pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.737231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.737210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4d41a76-0f08-4a04-bbbe-a96749e38076-tls-certs\") pod \"kube-auth-proxy-5f459d4668-2qzl2\" (UID: \"c4d41a76-0f08-4a04-bbbe-a96749e38076\") " pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.838512 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.838476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crzgp\" (UniqueName: \"kubernetes.io/projected/c4d41a76-0f08-4a04-bbbe-a96749e38076-kube-api-access-crzgp\") pod \"kube-auth-proxy-5f459d4668-2qzl2\" (UID: \"c4d41a76-0f08-4a04-bbbe-a96749e38076\") " pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.838512 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.838514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4d41a76-0f08-4a04-bbbe-a96749e38076-tls-certs\") pod \"kube-auth-proxy-5f459d4668-2qzl2\" (UID: \"c4d41a76-0f08-4a04-bbbe-a96749e38076\") " pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.838780 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.838548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4d41a76-0f08-4a04-bbbe-a96749e38076-tmp\") pod \"kube-auth-proxy-5f459d4668-2qzl2\" (UID: \"c4d41a76-0f08-4a04-bbbe-a96749e38076\") " pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.840972 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.840943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4d41a76-0f08-4a04-bbbe-a96749e38076-tmp\") pod \"kube-auth-proxy-5f459d4668-2qzl2\" (UID: \"c4d41a76-0f08-4a04-bbbe-a96749e38076\") " pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.841181 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.841164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4d41a76-0f08-4a04-bbbe-a96749e38076-tls-certs\") pod \"kube-auth-proxy-5f459d4668-2qzl2\" (UID: \"c4d41a76-0f08-4a04-bbbe-a96749e38076\") " pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.872467 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.872435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crzgp\" (UniqueName: \"kubernetes.io/projected/c4d41a76-0f08-4a04-bbbe-a96749e38076-kube-api-access-crzgp\") pod \"kube-auth-proxy-5f459d4668-2qzl2\" (UID: \"c4d41a76-0f08-4a04-bbbe-a96749e38076\") " pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:32.975603 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:32.975558    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"
Jul 24 03:26:33.189970 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:33.189942    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2"]
Jul 24 03:26:33.191397 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:26:33.191370    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d41a76_0f08_4a04_bbbe_a96749e38076.slice/crio-98015c99c5409a6cbf5492f2d87063d5aab342b7b752eeaddcc79d3e90963f02 WatchSource:0}: Error finding container 98015c99c5409a6cbf5492f2d87063d5aab342b7b752eeaddcc79d3e90963f02: Status 404 returned error can't find the container with id 98015c99c5409a6cbf5492f2d87063d5aab342b7b752eeaddcc79d3e90963f02
Jul 24 03:26:33.709790 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:33.709746    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2" event={"ID":"c4d41a76-0f08-4a04-bbbe-a96749e38076","Type":"ContainerStarted","Data":"98015c99c5409a6cbf5492f2d87063d5aab342b7b752eeaddcc79d3e90963f02"}
Jul 24 03:26:36.727482 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:36.727442    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2" event={"ID":"c4d41a76-0f08-4a04-bbbe-a96749e38076","Type":"ContainerStarted","Data":"4f285d98363fdf4d2cacf5e934807d54bd0ff4b23504b0895bf0cc03125683f4"}
Jul 24 03:26:36.754981 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:36.754923    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f459d4668-2qzl2" podStartSLOduration=2.177315882 podStartE2EDuration="4.754905417s" totalImagesPullingTime="2.577589535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:26:36.754665101 +0000 UTC m=+563.515573582" watchObservedRunningTime="2026-07-24 03:26:36.754905417 +0000 UTC m=+563.515813906"
Jul 24 03:26:42.714965 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.714915    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"]
Jul 24 03:26:42.718404 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.718381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.728140 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.728073    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:26:42.728369 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.728354    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 03:26:42.728439 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.728357    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 03:26:42.728869 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.728848    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 03:26:42.729015 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.729000    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-47bb4\""
Jul 24 03:26:42.730908 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.730893    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 03:26:42.741445 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.741409    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"]
Jul 24 03:26:42.822135 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.822092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-metrics-cert\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.822135 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.822138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-cert\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.822383 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.822199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-manager-config\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.822383 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.822231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spcqg\" (UniqueName: \"kubernetes.io/projected/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-kube-api-access-spcqg\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.922919 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.922878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-metrics-cert\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.922919 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.922928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-cert\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.923178 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.922981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-manager-config\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.923178 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.923014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spcqg\" (UniqueName: \"kubernetes.io/projected/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-kube-api-access-spcqg\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.923730 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.923706    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-manager-config\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.925685 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.925661    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-cert\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.925779 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.925691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-metrics-cert\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:42.936854 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:42.936819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spcqg\" (UniqueName: \"kubernetes.io/projected/008a9dec-e4ec-46e3-9f74-1d60d68b5cee-kube-api-access-spcqg\") pod \"lws-controller-manager-54698f7c78-rrv9j\" (UID: \"008a9dec-e4ec-46e3-9f74-1d60d68b5cee\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:43.027786 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:43.027695    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:43.175374 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:43.175344    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"]
Jul 24 03:26:43.176023 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:26:43.175996    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod008a9dec_e4ec_46e3_9f74_1d60d68b5cee.slice/crio-d5f5b825f0710bed9b593ccf366fb14cb3a1c1119c6e0b5566e17712e81bed48 WatchSource:0}: Error finding container d5f5b825f0710bed9b593ccf366fb14cb3a1c1119c6e0b5566e17712e81bed48: Status 404 returned error can't find the container with id d5f5b825f0710bed9b593ccf366fb14cb3a1c1119c6e0b5566e17712e81bed48
Jul 24 03:26:43.759701 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:43.759662    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j" event={"ID":"008a9dec-e4ec-46e3-9f74-1d60d68b5cee","Type":"ContainerStarted","Data":"d5f5b825f0710bed9b593ccf366fb14cb3a1c1119c6e0b5566e17712e81bed48"}
Jul 24 03:26:44.766225 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:44.766182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j" event={"ID":"008a9dec-e4ec-46e3-9f74-1d60d68b5cee","Type":"ContainerStarted","Data":"dc2154251b4ebae8015cf9546d1af5a7fcb3d328a2390b77094515485e8b9fd9"}
Jul 24 03:26:44.766651 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:44.766325    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:26:44.831564 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:44.831464    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j" podStartSLOduration=1.485145105 podStartE2EDuration="2.831442211s" totalImagesPullingTime="1.346297106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:26:44.830343822 +0000 UTC m=+571.591252301" watchObservedRunningTime="2026-07-24 03:26:44.831442211 +0000 UTC m=+571.592350691"
Jul 24 03:26:55.772415 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:26:55.772382    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-rrv9j"
Jul 24 03:27:48.970034 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:48.969949    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"]
Jul 24 03:27:48.973312 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:48.973296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:48.975560 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:48.975540    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 03:27:48.976294 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:48.976280    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4hl77\""
Jul 24 03:27:48.976354 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:48.976304    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 03:27:48.981297 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:48.981256    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"]
Jul 24 03:27:49.074587 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.074554    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.074790 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.074604    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n92fb\" (UniqueName: \"kubernetes.io/projected/841c14d6-e79e-46a9-b8fc-c03df3568324-kube-api-access-n92fb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.074790 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.074709    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.175912 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.175875    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n92fb\" (UniqueName: \"kubernetes.io/projected/841c14d6-e79e-46a9-b8fc-c03df3568324-kube-api-access-n92fb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.176085 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.175935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.176085 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.175968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.176328 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.176310    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.176401 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.176334    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.184348 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.184319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n92fb\" (UniqueName: \"kubernetes.io/projected/841c14d6-e79e-46a9-b8fc-c03df3568324-kube-api-access-n92fb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.283604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.283523    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:49.415370 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.415343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"]
Jul 24 03:27:49.417457 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:27:49.417430    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841c14d6_e79e_46a9_b8fc_c03df3568324.slice/crio-fe103b2b75ac100720b2475ccee34e01c0e3e44998b10eca0d075513e09f66ea WatchSource:0}: Error finding container fe103b2b75ac100720b2475ccee34e01c0e3e44998b10eca0d075513e09f66ea: Status 404 returned error can't find the container with id fe103b2b75ac100720b2475ccee34e01c0e3e44998b10eca0d075513e09f66ea
Jul 24 03:27:49.570670 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.570589    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"]
Jul 24 03:27:49.573906 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.573889    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.582014 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.581990    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"]
Jul 24 03:27:49.681417 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.681382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjvxb\" (UniqueName: \"kubernetes.io/projected/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-kube-api-access-pjvxb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.681417 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.681418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.681646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.681498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.781978 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.781947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.782147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.782006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjvxb\" (UniqueName: \"kubernetes.io/projected/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-kube-api-access-pjvxb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.782147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.782131    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.782811 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.782751    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.782811 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.782757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.790711 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.790688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjvxb\" (UniqueName: \"kubernetes.io/projected/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-kube-api-access-pjvxb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:49.883799 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:49.883709    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:50.014519 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.014487    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"]
Jul 24 03:27:50.017845 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:27:50.017813    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690f73ef_e7a6_4e4f_876e_ba9c124b4f5a.slice/crio-1650c118c44428d0d7a7643d87462d120723634df18ac4a0a22ee438988e4ed9 WatchSource:0}: Error finding container 1650c118c44428d0d7a7643d87462d120723634df18ac4a0a22ee438988e4ed9: Status 404 returned error can't find the container with id 1650c118c44428d0d7a7643d87462d120723634df18ac4a0a22ee438988e4ed9
Jul 24 03:27:50.056945 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.056914    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8" event={"ID":"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a","Type":"ContainerStarted","Data":"1650c118c44428d0d7a7643d87462d120723634df18ac4a0a22ee438988e4ed9"}
Jul 24 03:27:50.058495 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.058468    2577 generic.go:356] "Generic (PLEG): container finished" podID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerID="1e580827ba39a38c10aa5dd52c582d7f5a4c299642b8a54cfc8b43b63abacd4c" exitCode=0
Jul 24 03:27:50.058625 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.058530    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls" event={"ID":"841c14d6-e79e-46a9-b8fc-c03df3568324","Type":"ContainerDied","Data":"1e580827ba39a38c10aa5dd52c582d7f5a4c299642b8a54cfc8b43b63abacd4c"}
Jul 24 03:27:50.058625 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.058561    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls" event={"ID":"841c14d6-e79e-46a9-b8fc-c03df3568324","Type":"ContainerStarted","Data":"fe103b2b75ac100720b2475ccee34e01c0e3e44998b10eca0d075513e09f66ea"}
Jul 24 03:27:50.168661 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.168629    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"]
Jul 24 03:27:50.172036 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.172020    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.179246 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.179219    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"]
Jul 24 03:27:50.286905 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.286863    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.286905 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.286901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.287109 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.286929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv5bm\" (UniqueName: \"kubernetes.io/projected/0bd705fb-0c13-4616-ba19-d3e6b9776854-kube-api-access-gv5bm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.387994 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.387937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.387994 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.387995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.388231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.388041    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5bm\" (UniqueName: \"kubernetes.io/projected/0bd705fb-0c13-4616-ba19-d3e6b9776854-kube-api-access-gv5bm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.388453 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.388429    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.388488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.388435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.396319 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.396289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv5bm\" (UniqueName: \"kubernetes.io/projected/0bd705fb-0c13-4616-ba19-d3e6b9776854-kube-api-access-gv5bm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.500924 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.500833    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:50.567322 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.567290    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"]
Jul 24 03:27:50.570605 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.570578    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.578834 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.578793    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"]
Jul 24 03:27:50.647111 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.647085    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"]
Jul 24 03:27:50.690384 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.690332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.690567 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.690545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgml6\" (UniqueName: \"kubernetes.io/projected/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-kube-api-access-kgml6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.690630 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.690599    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.715375 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:27:50.715339    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd705fb_0c13_4616_ba19_d3e6b9776854.slice/crio-78c73da4dccfdde1fcf86845f4cc07d95338cc3b0ab84a2f32026cf6ba0c5855 WatchSource:0}: Error finding container 78c73da4dccfdde1fcf86845f4cc07d95338cc3b0ab84a2f32026cf6ba0c5855: Status 404 returned error can't find the container with id 78c73da4dccfdde1fcf86845f4cc07d95338cc3b0ab84a2f32026cf6ba0c5855
Jul 24 03:27:50.791700 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.791669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgml6\" (UniqueName: \"kubernetes.io/projected/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-kube-api-access-kgml6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.791880 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.791711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.791880 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.791733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.792132 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.792110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.792169 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.792119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.801457 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.801423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgml6\" (UniqueName: \"kubernetes.io/projected/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-kube-api-access-kgml6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:50.882644 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:50.882612    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:51.012717 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:51.012686    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"]
Jul 24 03:27:51.014349 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:27:51.014326    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3dd54b4_9deb_4ad6_bd3c_ab77f710ab9b.slice/crio-ab03d0de2aee95a81b5ac4bfe3a4fb2478440a4584c7ee7f05fbd0f57e4b84bc WatchSource:0}: Error finding container ab03d0de2aee95a81b5ac4bfe3a4fb2478440a4584c7ee7f05fbd0f57e4b84bc: Status 404 returned error can't find the container with id ab03d0de2aee95a81b5ac4bfe3a4fb2478440a4584c7ee7f05fbd0f57e4b84bc
Jul 24 03:27:51.065080 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:51.065050    2577 generic.go:356] "Generic (PLEG): container finished" podID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerID="08db123c0f948f1551585dcb3ccc8a8a6c9a1faa356c19485e3d0ff59cffd144" exitCode=0
Jul 24 03:27:51.065517 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:51.065134    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8" event={"ID":"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a","Type":"ContainerDied","Data":"08db123c0f948f1551585dcb3ccc8a8a6c9a1faa356c19485e3d0ff59cffd144"}
Jul 24 03:27:51.066958 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:51.066935    2577 generic.go:356] "Generic (PLEG): container finished" podID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerID="14bd99c76d8402433d35489791d1a58d34c5af23a403f09321fdb98952417b42" exitCode=0
Jul 24 03:27:51.067052 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:51.067016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls" event={"ID":"841c14d6-e79e-46a9-b8fc-c03df3568324","Type":"ContainerDied","Data":"14bd99c76d8402433d35489791d1a58d34c5af23a403f09321fdb98952417b42"}
Jul 24 03:27:51.068751 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:51.068649    2577 generic.go:356] "Generic (PLEG): container finished" podID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerID="74a993c86a8b42bd2fc0a8828d6e81104ce41a11f42499710ded172627aabaa2" exitCode=0
Jul 24 03:27:51.068809 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:51.068748    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29" event={"ID":"0bd705fb-0c13-4616-ba19-d3e6b9776854","Type":"ContainerDied","Data":"74a993c86a8b42bd2fc0a8828d6e81104ce41a11f42499710ded172627aabaa2"}
Jul 24 03:27:51.068809 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:51.068786    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29" event={"ID":"0bd705fb-0c13-4616-ba19-d3e6b9776854","Type":"ContainerStarted","Data":"78c73da4dccfdde1fcf86845f4cc07d95338cc3b0ab84a2f32026cf6ba0c5855"}
Jul 24 03:27:51.070601 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:51.070551    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw" event={"ID":"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b","Type":"ContainerStarted","Data":"ab03d0de2aee95a81b5ac4bfe3a4fb2478440a4584c7ee7f05fbd0f57e4b84bc"}
Jul 24 03:27:52.077607 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:52.077514    2577 generic.go:356] "Generic (PLEG): container finished" podID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerID="ae4b4f56e5640ba9b9ead78d9d1ae897161660a0939410b44737ba6624db6f8c" exitCode=0
Jul 24 03:27:52.078014 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:52.077600    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8" event={"ID":"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a","Type":"ContainerDied","Data":"ae4b4f56e5640ba9b9ead78d9d1ae897161660a0939410b44737ba6624db6f8c"}
Jul 24 03:27:52.079803 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:52.079780    2577 generic.go:356] "Generic (PLEG): container finished" podID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerID="cbbf41ad11bb93515fe502c713b32f2a94a191aa94a77182ae49f937a582ca4c" exitCode=0
Jul 24 03:27:52.079899 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:52.079855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls" event={"ID":"841c14d6-e79e-46a9-b8fc-c03df3568324","Type":"ContainerDied","Data":"cbbf41ad11bb93515fe502c713b32f2a94a191aa94a77182ae49f937a582ca4c"}
Jul 24 03:27:52.081633 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:52.081612    2577 generic.go:356] "Generic (PLEG): container finished" podID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerID="6054d563562d336f3be80792ff08a6764b150f92a6eb0beeebebe17e262363ca" exitCode=0
Jul 24 03:27:52.081745 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:52.081721    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29" event={"ID":"0bd705fb-0c13-4616-ba19-d3e6b9776854","Type":"ContainerDied","Data":"6054d563562d336f3be80792ff08a6764b150f92a6eb0beeebebe17e262363ca"}
Jul 24 03:27:52.083478 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:52.083191    2577 generic.go:356] "Generic (PLEG): container finished" podID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerID="1245abe0aa40159d893bbccae625e8bb893e5942679776bcaecf61fd9f91fd67" exitCode=0
Jul 24 03:27:52.083478 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:52.083229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw" event={"ID":"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b","Type":"ContainerDied","Data":"1245abe0aa40159d893bbccae625e8bb893e5942679776bcaecf61fd9f91fd67"}
Jul 24 03:27:53.089731 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.089638    2577 generic.go:356] "Generic (PLEG): container finished" podID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerID="b95ef2fc51950c91c4d5b1245432c9e8e39ae56b58a2a3ec9e760b2d816bf5ed" exitCode=0
Jul 24 03:27:53.090148 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.089731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29" event={"ID":"0bd705fb-0c13-4616-ba19-d3e6b9776854","Type":"ContainerDied","Data":"b95ef2fc51950c91c4d5b1245432c9e8e39ae56b58a2a3ec9e760b2d816bf5ed"}
Jul 24 03:27:53.091386 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.091364    2577 generic.go:356] "Generic (PLEG): container finished" podID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerID="61798c4dd9897857b87f3ea5133c1a8016d5611d20e2bf33555625f6b30b40c7" exitCode=0
Jul 24 03:27:53.091498 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.091441    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw" event={"ID":"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b","Type":"ContainerDied","Data":"61798c4dd9897857b87f3ea5133c1a8016d5611d20e2bf33555625f6b30b40c7"}
Jul 24 03:27:53.093781 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.093756    2577 generic.go:356] "Generic (PLEG): container finished" podID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerID="802299c1db13cc12fddb227a0379265ef96aad2ebf86e429ead78b37ef7a3ec9" exitCode=0
Jul 24 03:27:53.093860 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.093815    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8" event={"ID":"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a","Type":"ContainerDied","Data":"802299c1db13cc12fddb227a0379265ef96aad2ebf86e429ead78b37ef7a3ec9"}
Jul 24 03:27:53.232369 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.232341    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:53.281887 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:27:53.281851    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3dd54b4_9deb_4ad6_bd3c_ab77f710ab9b.slice/crio-conmon-31e4df67260dea2b7148caa85c87d7a803932b0d162b28ca901e48e1c7fb4f34.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 03:27:53.282010 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:27:53.281992    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3dd54b4_9deb_4ad6_bd3c_ab77f710ab9b.slice/crio-conmon-31e4df67260dea2b7148caa85c87d7a803932b0d162b28ca901e48e1c7fb4f34.scope\": RecentStats: unable to find data in memory cache]"
Jul 24 03:27:53.314850 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.314815    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n92fb\" (UniqueName: \"kubernetes.io/projected/841c14d6-e79e-46a9-b8fc-c03df3568324-kube-api-access-n92fb\") pod \"841c14d6-e79e-46a9-b8fc-c03df3568324\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") "
Jul 24 03:27:53.315011 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.314866    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-bundle\") pod \"841c14d6-e79e-46a9-b8fc-c03df3568324\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") "
Jul 24 03:27:53.315011 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.314947    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-util\") pod \"841c14d6-e79e-46a9-b8fc-c03df3568324\" (UID: \"841c14d6-e79e-46a9-b8fc-c03df3568324\") "
Jul 24 03:27:53.315480 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.315447    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-bundle" (OuterVolumeSpecName: "bundle") pod "841c14d6-e79e-46a9-b8fc-c03df3568324" (UID: "841c14d6-e79e-46a9-b8fc-c03df3568324"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:27:53.317115 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.317095    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/841c14d6-e79e-46a9-b8fc-c03df3568324-kube-api-access-n92fb" (OuterVolumeSpecName: "kube-api-access-n92fb") pod "841c14d6-e79e-46a9-b8fc-c03df3568324" (UID: "841c14d6-e79e-46a9-b8fc-c03df3568324"). InnerVolumeSpecName "kube-api-access-n92fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:27:53.320572 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.320548    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-util" (OuterVolumeSpecName: "util") pod "841c14d6-e79e-46a9-b8fc-c03df3568324" (UID: "841c14d6-e79e-46a9-b8fc-c03df3568324"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:27:53.416387 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.416360    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n92fb\" (UniqueName: \"kubernetes.io/projected/841c14d6-e79e-46a9-b8fc-c03df3568324-kube-api-access-n92fb\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:53.416387 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.416388    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:53.416574 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:53.416398    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/841c14d6-e79e-46a9-b8fc-c03df3568324-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:54.100727 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.100693    2577 generic.go:356] "Generic (PLEG): container finished" podID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerID="31e4df67260dea2b7148caa85c87d7a803932b0d162b28ca901e48e1c7fb4f34" exitCode=0
Jul 24 03:27:54.101146 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.100773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw" event={"ID":"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b","Type":"ContainerDied","Data":"31e4df67260dea2b7148caa85c87d7a803932b0d162b28ca901e48e1c7fb4f34"}
Jul 24 03:27:54.102607 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.102583    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls"
Jul 24 03:27:54.102738 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.102611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls" event={"ID":"841c14d6-e79e-46a9-b8fc-c03df3568324","Type":"ContainerDied","Data":"fe103b2b75ac100720b2475ccee34e01c0e3e44998b10eca0d075513e09f66ea"}
Jul 24 03:27:54.102738 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.102653    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe103b2b75ac100720b2475ccee34e01c0e3e44998b10eca0d075513e09f66ea"
Jul 24 03:27:54.257464 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.257437    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:54.259495 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.259477    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:54.324338 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.324309    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv5bm\" (UniqueName: \"kubernetes.io/projected/0bd705fb-0c13-4616-ba19-d3e6b9776854-kube-api-access-gv5bm\") pod \"0bd705fb-0c13-4616-ba19-d3e6b9776854\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") "
Jul 24 03:27:54.324504 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.324353    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-util\") pod \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") "
Jul 24 03:27:54.324504 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.324392    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-bundle\") pod \"0bd705fb-0c13-4616-ba19-d3e6b9776854\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") "
Jul 24 03:27:54.324504 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.324412    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-util\") pod \"0bd705fb-0c13-4616-ba19-d3e6b9776854\" (UID: \"0bd705fb-0c13-4616-ba19-d3e6b9776854\") "
Jul 24 03:27:54.324504 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.324452    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-bundle\") pod \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") "
Jul 24 03:27:54.324504 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.324477    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjvxb\" (UniqueName: \"kubernetes.io/projected/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-kube-api-access-pjvxb\") pod \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\" (UID: \"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a\") "
Jul 24 03:27:54.325432 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.325386    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-bundle" (OuterVolumeSpecName: "bundle") pod "0bd705fb-0c13-4616-ba19-d3e6b9776854" (UID: "0bd705fb-0c13-4616-ba19-d3e6b9776854"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:27:54.325552 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.325490    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-bundle" (OuterVolumeSpecName: "bundle") pod "690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" (UID: "690f73ef-e7a6-4e4f-876e-ba9c124b4f5a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:27:54.326741 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.326715    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bd705fb-0c13-4616-ba19-d3e6b9776854-kube-api-access-gv5bm" (OuterVolumeSpecName: "kube-api-access-gv5bm") pod "0bd705fb-0c13-4616-ba19-d3e6b9776854" (UID: "0bd705fb-0c13-4616-ba19-d3e6b9776854"). InnerVolumeSpecName "kube-api-access-gv5bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:27:54.326988 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.326971    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-kube-api-access-pjvxb" (OuterVolumeSpecName: "kube-api-access-pjvxb") pod "690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" (UID: "690f73ef-e7a6-4e4f-876e-ba9c124b4f5a"). InnerVolumeSpecName "kube-api-access-pjvxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:27:54.330702 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.330678    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-util" (OuterVolumeSpecName: "util") pod "690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" (UID: "690f73ef-e7a6-4e4f-876e-ba9c124b4f5a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:27:54.330950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.330926    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-util" (OuterVolumeSpecName: "util") pod "0bd705fb-0c13-4616-ba19-d3e6b9776854" (UID: "0bd705fb-0c13-4616-ba19-d3e6b9776854"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:27:54.425767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.425738    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv5bm\" (UniqueName: \"kubernetes.io/projected/0bd705fb-0c13-4616-ba19-d3e6b9776854-kube-api-access-gv5bm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:54.425767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.425768    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:54.425937 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.425778    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:54.425937 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.425786    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0bd705fb-0c13-4616-ba19-d3e6b9776854-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:54.425937 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.425794    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:54.425937 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:54.425801    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjvxb\" (UniqueName: \"kubernetes.io/projected/690f73ef-e7a6-4e4f-876e-ba9c124b4f5a-kube-api-access-pjvxb\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:55.109755 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.109723    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29"
Jul 24 03:27:55.109755 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.109744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29" event={"ID":"0bd705fb-0c13-4616-ba19-d3e6b9776854","Type":"ContainerDied","Data":"78c73da4dccfdde1fcf86845f4cc07d95338cc3b0ab84a2f32026cf6ba0c5855"}
Jul 24 03:27:55.110251 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.109799    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78c73da4dccfdde1fcf86845f4cc07d95338cc3b0ab84a2f32026cf6ba0c5855"
Jul 24 03:27:55.111660 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.111635    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8"
Jul 24 03:27:55.111798 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.111662    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8" event={"ID":"690f73ef-e7a6-4e4f-876e-ba9c124b4f5a","Type":"ContainerDied","Data":"1650c118c44428d0d7a7643d87462d120723634df18ac4a0a22ee438988e4ed9"}
Jul 24 03:27:55.111798 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.111700    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1650c118c44428d0d7a7643d87462d120723634df18ac4a0a22ee438988e4ed9"
Jul 24 03:27:55.232572 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.232542    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:55.333624 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.333578    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgml6\" (UniqueName: \"kubernetes.io/projected/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-kube-api-access-kgml6\") pod \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") "
Jul 24 03:27:55.333828 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.333680    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-bundle\") pod \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") "
Jul 24 03:27:55.333828 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.333760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-util\") pod \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\" (UID: \"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b\") "
Jul 24 03:27:55.334257 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.334226    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-bundle" (OuterVolumeSpecName: "bundle") pod "a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" (UID: "a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:27:55.335822 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.335800    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-kube-api-access-kgml6" (OuterVolumeSpecName: "kube-api-access-kgml6") pod "a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" (UID: "a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b"). InnerVolumeSpecName "kube-api-access-kgml6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:27:55.341781 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.341753    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-util" (OuterVolumeSpecName: "util") pod "a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" (UID: "a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:27:55.435135 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.435100    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-util\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:55.435135 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.435129    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgml6\" (UniqueName: \"kubernetes.io/projected/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-kube-api-access-kgml6\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:55.435135 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:55.435142    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:27:56.118032 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:56.118001    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw"
Jul 24 03:27:56.118032 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:56.118027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw" event={"ID":"a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b","Type":"ContainerDied","Data":"ab03d0de2aee95a81b5ac4bfe3a4fb2478440a4584c7ee7f05fbd0f57e4b84bc"}
Jul 24 03:27:56.118477 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:27:56.118071    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab03d0de2aee95a81b5ac4bfe3a4fb2478440a4584c7ee7f05fbd0f57e4b84bc"
Jul 24 03:28:46.091231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091197    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6447457645-26pxx"]
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091514    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerName="pull"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091526    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerName="pull"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091537    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerName="extract"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091543    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerName="extract"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091557    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerName="extract"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091563    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerName="extract"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091570    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerName="extract"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091575    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerName="extract"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091580    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerName="util"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091585    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerName="util"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091590    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerName="util"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091595    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerName="util"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091601    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerName="util"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091605    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerName="util"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091613    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerName="pull"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091617    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerName="pull"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091622    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerName="extract"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091627    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerName="extract"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091634    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerName="util"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091639    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerName="util"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091644    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerName="pull"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091649    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerName="pull"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091656    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerName="pull"
Jul 24 03:28:46.091681 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091660    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerName="pull"
Jul 24 03:28:46.092381 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091712    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b" containerName="extract"
Jul 24 03:28:46.092381 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091722    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="841c14d6-e79e-46a9-b8fc-c03df3568324" containerName="extract"
Jul 24 03:28:46.092381 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091729    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="690f73ef-e7a6-4e4f-876e-ba9c124b4f5a" containerName="extract"
Jul 24 03:28:46.092381 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.091735    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bd705fb-0c13-4616-ba19-d3e6b9776854" containerName="extract"
Jul 24 03:28:46.094775 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.094758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.109573 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.109537    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6447457645-26pxx"]
Jul 24 03:28:46.160318 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.160284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-console-serving-cert\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.160521 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.160356    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-trusted-ca-bundle\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.160521 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.160431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-console-oauth-config\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.160521 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.160475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-oauth-serving-cert\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.160521 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.160501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64xgn\" (UniqueName: \"kubernetes.io/projected/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-kube-api-access-64xgn\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.160688 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.160529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-console-config\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.160688 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.160577    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-service-ca\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.261376 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.261340    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-trusted-ca-bundle\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.261376 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.261382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-console-oauth-config\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.261577 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.261399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-oauth-serving-cert\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.261577 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.261417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64xgn\" (UniqueName: \"kubernetes.io/projected/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-kube-api-access-64xgn\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.261577 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.261440    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-console-config\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.261577 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.261482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-service-ca\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.261577 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.261524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-console-serving-cert\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.262323 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.262294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-console-config\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.262429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.262336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-service-ca\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.262429 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.262383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-trusted-ca-bundle\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.262582 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.262561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-oauth-serving-cert\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.263951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.263929    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-console-serving-cert\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.264114 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.264093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-console-oauth-config\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.268921 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.268894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64xgn\" (UniqueName: \"kubernetes.io/projected/ad74ecb7-3c84-4ee1-965d-10686ed5eddd-kube-api-access-64xgn\") pod \"console-6447457645-26pxx\" (UID: \"ad74ecb7-3c84-4ee1-965d-10686ed5eddd\") " pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.403619 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.403530    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:46.558503 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:46.558477    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6447457645-26pxx"]
Jul 24 03:28:46.560636 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:28:46.560606    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad74ecb7_3c84_4ee1_965d_10686ed5eddd.slice/crio-a84599f05ec119de1a5795dd1c2e86cda7595426a077dc8ce44f7977a6a629e7 WatchSource:0}: Error finding container a84599f05ec119de1a5795dd1c2e86cda7595426a077dc8ce44f7977a6a629e7: Status 404 returned error can't find the container with id a84599f05ec119de1a5795dd1c2e86cda7595426a077dc8ce44f7977a6a629e7
Jul 24 03:28:47.361177 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:47.361145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6447457645-26pxx" event={"ID":"ad74ecb7-3c84-4ee1-965d-10686ed5eddd","Type":"ContainerStarted","Data":"12d5be64fd43fdf53bb5adb2a207a44f0d1bc866b6395b7b6200ddb087f2cbd5"}
Jul 24 03:28:47.361597 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:47.361194    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6447457645-26pxx" event={"ID":"ad74ecb7-3c84-4ee1-965d-10686ed5eddd","Type":"ContainerStarted","Data":"a84599f05ec119de1a5795dd1c2e86cda7595426a077dc8ce44f7977a6a629e7"}
Jul 24 03:28:47.380440 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:47.380388    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6447457645-26pxx" podStartSLOduration=1.380371922 podStartE2EDuration="1.380371922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:28:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:28:47.378870969 +0000 UTC m=+694.139779451" watchObservedRunningTime="2026-07-24 03:28:47.380371922 +0000 UTC m=+694.141280401"
Jul 24 03:28:56.403936 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:56.403901    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:56.404388 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:56.403957    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:56.408631 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:56.408609    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:57.411428 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:57.411401    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6447457645-26pxx"
Jul 24 03:28:57.456725 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:28:57.456692    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5bc97f9d-wpfck"]
Jul 24 03:29:08.291371 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.291301    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:29:08.294439 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.294421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:08.297743 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.297726    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4hl77\""
Jul 24 03:29:08.298244 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.298228    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 03:29:08.302544 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.302525    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 03:29:08.302614 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.302525    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 03:29:08.312172 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.312142    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:29:08.349284 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.349233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5plg\" (UniqueName: \"kubernetes.io/projected/5b8302a0-c23a-4e3c-af84-8f77b50dbcb6-kube-api-access-f5plg\") pod \"limitador-limitador-54d6d988d5-pv8k4\" (UID: \"5b8302a0-c23a-4e3c-af84-8f77b50dbcb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:08.349427 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.349322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b8302a0-c23a-4e3c-af84-8f77b50dbcb6-config-file\") pod \"limitador-limitador-54d6d988d5-pv8k4\" (UID: \"5b8302a0-c23a-4e3c-af84-8f77b50dbcb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:08.387589 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.387553    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:29:08.449897 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.449858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5plg\" (UniqueName: \"kubernetes.io/projected/5b8302a0-c23a-4e3c-af84-8f77b50dbcb6-kube-api-access-f5plg\") pod \"limitador-limitador-54d6d988d5-pv8k4\" (UID: \"5b8302a0-c23a-4e3c-af84-8f77b50dbcb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:08.450087 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.449977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b8302a0-c23a-4e3c-af84-8f77b50dbcb6-config-file\") pod \"limitador-limitador-54d6d988d5-pv8k4\" (UID: \"5b8302a0-c23a-4e3c-af84-8f77b50dbcb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:08.450618 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.450593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b8302a0-c23a-4e3c-af84-8f77b50dbcb6-config-file\") pod \"limitador-limitador-54d6d988d5-pv8k4\" (UID: \"5b8302a0-c23a-4e3c-af84-8f77b50dbcb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:08.457918 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.457891    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5plg\" (UniqueName: \"kubernetes.io/projected/5b8302a0-c23a-4e3c-af84-8f77b50dbcb6-kube-api-access-f5plg\") pod \"limitador-limitador-54d6d988d5-pv8k4\" (UID: \"5b8302a0-c23a-4e3c-af84-8f77b50dbcb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:08.604629 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.604545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:08.735516 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.735491    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:29:08.737669 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:29:08.737631    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8302a0_c23a_4e3c_af84_8f77b50dbcb6.slice/crio-59eeb221246be503762d08b9d0fafe14e90f0c641b63babc61ac75ac18328bf5 WatchSource:0}: Error finding container 59eeb221246be503762d08b9d0fafe14e90f0c641b63babc61ac75ac18328bf5: Status 404 returned error can't find the container with id 59eeb221246be503762d08b9d0fafe14e90f0c641b63babc61ac75ac18328bf5
Jul 24 03:29:08.739771 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.739755    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 03:29:08.887418 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.887344    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-rpkpm"]
Jul 24 03:29:08.891782 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.891764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rpkpm"
Jul 24 03:29:08.894346 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.894294    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wxnqx\""
Jul 24 03:29:08.896344 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.896324    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rpkpm"]
Jul 24 03:29:08.953472 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:08.953429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdgzk\" (UniqueName: \"kubernetes.io/projected/4e99bff8-3f68-4d24-b1cb-a6e7050bfac2-kube-api-access-vdgzk\") pod \"authorino-75d556bd77-rpkpm\" (UID: \"4e99bff8-3f68-4d24-b1cb-a6e7050bfac2\") " pod="kuadrant-system/authorino-75d556bd77-rpkpm"
Jul 24 03:29:09.054513 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:09.054473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdgzk\" (UniqueName: \"kubernetes.io/projected/4e99bff8-3f68-4d24-b1cb-a6e7050bfac2-kube-api-access-vdgzk\") pod \"authorino-75d556bd77-rpkpm\" (UID: \"4e99bff8-3f68-4d24-b1cb-a6e7050bfac2\") " pod="kuadrant-system/authorino-75d556bd77-rpkpm"
Jul 24 03:29:09.062866 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:09.062836    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdgzk\" (UniqueName: \"kubernetes.io/projected/4e99bff8-3f68-4d24-b1cb-a6e7050bfac2-kube-api-access-vdgzk\") pod \"authorino-75d556bd77-rpkpm\" (UID: \"4e99bff8-3f68-4d24-b1cb-a6e7050bfac2\") " pod="kuadrant-system/authorino-75d556bd77-rpkpm"
Jul 24 03:29:09.201549 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:09.201512    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rpkpm"
Jul 24 03:29:09.368919 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:09.368829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rpkpm"]
Jul 24 03:29:09.372813 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:29:09.372778    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e99bff8_3f68_4d24_b1cb_a6e7050bfac2.slice/crio-ac3f55b9f26bcc12b37a670a7b66244d7c3097b404ced420b03935c632024b8f WatchSource:0}: Error finding container ac3f55b9f26bcc12b37a670a7b66244d7c3097b404ced420b03935c632024b8f: Status 404 returned error can't find the container with id ac3f55b9f26bcc12b37a670a7b66244d7c3097b404ced420b03935c632024b8f
Jul 24 03:29:09.471879 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:09.471776    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4" event={"ID":"5b8302a0-c23a-4e3c-af84-8f77b50dbcb6","Type":"ContainerStarted","Data":"59eeb221246be503762d08b9d0fafe14e90f0c641b63babc61ac75ac18328bf5"}
Jul 24 03:29:09.473401 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:09.473372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rpkpm" event={"ID":"4e99bff8-3f68-4d24-b1cb-a6e7050bfac2","Type":"ContainerStarted","Data":"ac3f55b9f26bcc12b37a670a7b66244d7c3097b404ced420b03935c632024b8f"}
Jul 24 03:29:11.490117 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:11.490080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4" event={"ID":"5b8302a0-c23a-4e3c-af84-8f77b50dbcb6","Type":"ContainerStarted","Data":"cae5d876014fb41e6e1c462ff288aadf90f445d89d717f8793782d5362d5ef2d"}
Jul 24 03:29:11.490593 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:11.490210    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:11.506881 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:11.506831    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4" podStartSLOduration=0.971515613 podStartE2EDuration="3.506814177s" totalImagesPullingTime="2.535298564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:29:11.506278137 +0000 UTC m=+718.267186613" watchObservedRunningTime="2026-07-24 03:29:11.506814177 +0000 UTC m=+718.267722658"
Jul 24 03:29:13.503593 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:13.503553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rpkpm" event={"ID":"4e99bff8-3f68-4d24-b1cb-a6e7050bfac2","Type":"ContainerStarted","Data":"da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500"}
Jul 24 03:29:13.518860 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:13.518809    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-rpkpm" podStartSLOduration=1.7758089049999999 podStartE2EDuration="5.518792105s" totalImagesPullingTime="3.7429832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:29:13.517706232 +0000 UTC m=+720.278614711" watchObservedRunningTime="2026-07-24 03:29:13.518792105 +0000 UTC m=+720.279700619"
Jul 24 03:29:22.479345 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.479279    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f5bc97f9d-wpfck" podUID="9a77a591-df33-4d69-964e-c2e8d390e8df" containerName="console" containerID="cri-o://ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f" gracePeriod=15
Jul 24 03:29:22.498341 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.498319    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"
Jul 24 03:29:22.724972 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.724943    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:29:22.776699 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.776618    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-oauth-config\") pod \"9a77a591-df33-4d69-964e-c2e8d390e8df\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") "
Jul 24 03:29:22.776699 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.776682    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-oauth-serving-cert\") pod \"9a77a591-df33-4d69-964e-c2e8d390e8df\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") "
Jul 24 03:29:22.776903 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.776729    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzkht\" (UniqueName: \"kubernetes.io/projected/9a77a591-df33-4d69-964e-c2e8d390e8df-kube-api-access-xzkht\") pod \"9a77a591-df33-4d69-964e-c2e8d390e8df\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") "
Jul 24 03:29:22.776903 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.776784    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-console-config\") pod \"9a77a591-df33-4d69-964e-c2e8d390e8df\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") "
Jul 24 03:29:22.776903 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.776837    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-trusted-ca-bundle\") pod \"9a77a591-df33-4d69-964e-c2e8d390e8df\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") "
Jul 24 03:29:22.776903 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.776883    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-service-ca\") pod \"9a77a591-df33-4d69-964e-c2e8d390e8df\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") "
Jul 24 03:29:22.777101 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.776905    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-serving-cert\") pod \"9a77a591-df33-4d69-964e-c2e8d390e8df\" (UID: \"9a77a591-df33-4d69-964e-c2e8d390e8df\") "
Jul 24 03:29:22.777283 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.777235    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9a77a591-df33-4d69-964e-c2e8d390e8df" (UID: "9a77a591-df33-4d69-964e-c2e8d390e8df"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:29:22.777403 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.777381    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-console-config" (OuterVolumeSpecName: "console-config") pod "9a77a591-df33-4d69-964e-c2e8d390e8df" (UID: "9a77a591-df33-4d69-964e-c2e8d390e8df"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:29:22.777462 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.777397    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-service-ca" (OuterVolumeSpecName: "service-ca") pod "9a77a591-df33-4d69-964e-c2e8d390e8df" (UID: "9a77a591-df33-4d69-964e-c2e8d390e8df"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:29:22.777627 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.777605    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9a77a591-df33-4d69-964e-c2e8d390e8df" (UID: "9a77a591-df33-4d69-964e-c2e8d390e8df"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:29:22.779034 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.779006    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9a77a591-df33-4d69-964e-c2e8d390e8df" (UID: "9a77a591-df33-4d69-964e-c2e8d390e8df"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:29:22.779141 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.779081    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9a77a591-df33-4d69-964e-c2e8d390e8df" (UID: "9a77a591-df33-4d69-964e-c2e8d390e8df"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:29:22.779420 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.779403    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a77a591-df33-4d69-964e-c2e8d390e8df-kube-api-access-xzkht" (OuterVolumeSpecName: "kube-api-access-xzkht") pod "9a77a591-df33-4d69-964e-c2e8d390e8df" (UID: "9a77a591-df33-4d69-964e-c2e8d390e8df"). InnerVolumeSpecName "kube-api-access-xzkht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:29:22.877711 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.877674    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-console-config\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:22.877711 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.877703    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-trusted-ca-bundle\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:22.877711 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.877712    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:22.877711 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.877721    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-serving-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:22.877981 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.877729    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a77a591-df33-4d69-964e-c2e8d390e8df-console-oauth-config\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:22.877981 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.877738    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a77a591-df33-4d69-964e-c2e8d390e8df-oauth-serving-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:22.877981 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:22.877746    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzkht\" (UniqueName: \"kubernetes.io/projected/9a77a591-df33-4d69-964e-c2e8d390e8df-kube-api-access-xzkht\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:23.553492 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.553456    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a77a591-df33-4d69-964e-c2e8d390e8df" containerID="ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f" exitCode=0
Jul 24 03:29:23.553890 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.553503    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5bc97f9d-wpfck"
Jul 24 03:29:23.553890 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.553549    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5bc97f9d-wpfck" event={"ID":"9a77a591-df33-4d69-964e-c2e8d390e8df","Type":"ContainerDied","Data":"ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f"}
Jul 24 03:29:23.553890 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.553601    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5bc97f9d-wpfck" event={"ID":"9a77a591-df33-4d69-964e-c2e8d390e8df","Type":"ContainerDied","Data":"bed1850b7e8ecaed2bbf865d0b43098847692c2f23f6d5a9255c4e40018afb16"}
Jul 24 03:29:23.553890 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.553622    2577 scope.go:117] "RemoveContainer" containerID="ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f"
Jul 24 03:29:23.562491 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.562472    2577 scope.go:117] "RemoveContainer" containerID="ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f"
Jul 24 03:29:23.562741 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:29:23.562723    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f\": container with ID starting with ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f not found: ID does not exist" containerID="ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f"
Jul 24 03:29:23.562794 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.562757    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f"} err="failed to get container status \"ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f\": rpc error: code = NotFound desc = could not find container \"ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f\": container with ID starting with ebeb348919a952a1f0376f2fd693e5e270be47bd6fad21ab3a25457da7468c2f not found: ID does not exist"
Jul 24 03:29:23.576688 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.576665    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5bc97f9d-wpfck"]
Jul 24 03:29:23.582796 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.582769    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f5bc97f9d-wpfck"]
Jul 24 03:29:23.642037 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:23.642006    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a77a591-df33-4d69-964e-c2e8d390e8df" path="/var/lib/kubelet/pods/9a77a591-df33-4d69-964e-c2e8d390e8df/volumes"
Jul 24 03:29:36.577666 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.577631    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zzdp5"]
Jul 24 03:29:36.578028 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.577963    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a77a591-df33-4d69-964e-c2e8d390e8df" containerName="console"
Jul 24 03:29:36.578028 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.577974    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a77a591-df33-4d69-964e-c2e8d390e8df" containerName="console"
Jul 24 03:29:36.578028 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.578022    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a77a591-df33-4d69-964e-c2e8d390e8df" containerName="console"
Jul 24 03:29:36.582251 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.582233    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zzdp5"
Jul 24 03:29:36.588094 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.588067    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zzdp5"]
Jul 24 03:29:36.700006 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.699965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7gfn\" (UniqueName: \"kubernetes.io/projected/03f87bf6-cde3-4235-9302-fed144e4603a-kube-api-access-q7gfn\") pod \"authorino-5d4b67c446-zzdp5\" (UID: \"03f87bf6-cde3-4235-9302-fed144e4603a\") " pod="kuadrant-system/authorino-5d4b67c446-zzdp5"
Jul 24 03:29:36.801255 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.801222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7gfn\" (UniqueName: \"kubernetes.io/projected/03f87bf6-cde3-4235-9302-fed144e4603a-kube-api-access-q7gfn\") pod \"authorino-5d4b67c446-zzdp5\" (UID: \"03f87bf6-cde3-4235-9302-fed144e4603a\") " pod="kuadrant-system/authorino-5d4b67c446-zzdp5"
Jul 24 03:29:36.813618 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.813592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7gfn\" (UniqueName: \"kubernetes.io/projected/03f87bf6-cde3-4235-9302-fed144e4603a-kube-api-access-q7gfn\") pod \"authorino-5d4b67c446-zzdp5\" (UID: \"03f87bf6-cde3-4235-9302-fed144e4603a\") " pod="kuadrant-system/authorino-5d4b67c446-zzdp5"
Jul 24 03:29:36.825921 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.825886    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zzdp5"]
Jul 24 03:29:36.826123 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.826111    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zzdp5"
Jul 24 03:29:36.849933 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.849904    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bf7bdb849-hn584"]
Jul 24 03:29:36.854245 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.854222    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bf7bdb849-hn584"
Jul 24 03:29:36.859192 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.859163    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bf7bdb849-hn584"]
Jul 24 03:29:36.959742 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:36.959720    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zzdp5"]
Jul 24 03:29:36.961805 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:29:36.961778    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03f87bf6_cde3_4235_9302_fed144e4603a.slice/crio-a18cabcb833ec21c2a8175f4793fd34a1292f2ee9184ab608abc093f58c10ab1 WatchSource:0}: Error finding container a18cabcb833ec21c2a8175f4793fd34a1292f2ee9184ab608abc093f58c10ab1: Status 404 returned error can't find the container with id a18cabcb833ec21c2a8175f4793fd34a1292f2ee9184ab608abc093f58c10ab1
Jul 24 03:29:37.002999 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.002969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j8p9\" (UniqueName: \"kubernetes.io/projected/d19cb7d9-26c1-4965-a733-b3b360777b1a-kube-api-access-5j8p9\") pod \"authorino-7bf7bdb849-hn584\" (UID: \"d19cb7d9-26c1-4965-a733-b3b360777b1a\") " pod="kuadrant-system/authorino-7bf7bdb849-hn584"
Jul 24 03:29:37.104306 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.104197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j8p9\" (UniqueName: \"kubernetes.io/projected/d19cb7d9-26c1-4965-a733-b3b360777b1a-kube-api-access-5j8p9\") pod \"authorino-7bf7bdb849-hn584\" (UID: \"d19cb7d9-26c1-4965-a733-b3b360777b1a\") " pod="kuadrant-system/authorino-7bf7bdb849-hn584"
Jul 24 03:29:37.111975 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.111950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j8p9\" (UniqueName: \"kubernetes.io/projected/d19cb7d9-26c1-4965-a733-b3b360777b1a-kube-api-access-5j8p9\") pod \"authorino-7bf7bdb849-hn584\" (UID: \"d19cb7d9-26c1-4965-a733-b3b360777b1a\") " pod="kuadrant-system/authorino-7bf7bdb849-hn584"
Jul 24 03:29:37.164979 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.164942    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bf7bdb849-hn584"
Jul 24 03:29:37.295171 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.295143    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bf7bdb849-hn584"]
Jul 24 03:29:37.296950 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:29:37.296927    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd19cb7d9_26c1_4965_a733_b3b360777b1a.slice/crio-db3d0e00340385f992c078d1117ce0125ec5b48cc37f1702d12396c83d13964e WatchSource:0}: Error finding container db3d0e00340385f992c078d1117ce0125ec5b48cc37f1702d12396c83d13964e: Status 404 returned error can't find the container with id db3d0e00340385f992c078d1117ce0125ec5b48cc37f1702d12396c83d13964e
Jul 24 03:29:37.624343 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.624237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-zzdp5" event={"ID":"03f87bf6-cde3-4235-9302-fed144e4603a","Type":"ContainerStarted","Data":"c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d"}
Jul 24 03:29:37.624343 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.624257    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-zzdp5" podUID="03f87bf6-cde3-4235-9302-fed144e4603a" containerName="authorino" containerID="cri-o://c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d" gracePeriod=30
Jul 24 03:29:37.624343 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.624305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-zzdp5" event={"ID":"03f87bf6-cde3-4235-9302-fed144e4603a","Type":"ContainerStarted","Data":"a18cabcb833ec21c2a8175f4793fd34a1292f2ee9184ab608abc093f58c10ab1"}
Jul 24 03:29:37.625611 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.625584    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bf7bdb849-hn584" event={"ID":"d19cb7d9-26c1-4965-a733-b3b360777b1a","Type":"ContainerStarted","Data":"db3d0e00340385f992c078d1117ce0125ec5b48cc37f1702d12396c83d13964e"}
Jul 24 03:29:37.638406 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.638323    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-zzdp5" podStartSLOduration=1.240714197 podStartE2EDuration="1.638306352s" totalImagesPullingTime="397.592155ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:29:37.637582336 +0000 UTC m=+744.398490817" watchObservedRunningTime="2026-07-24 03:29:37.638306352 +0000 UTC m=+744.399214834"
Jul 24 03:29:37.688541 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.688507    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bf7bdb849-hn584"]
Jul 24 03:29:37.715680 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.715657    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c694979d9-trlhz"]
Jul 24 03:29:37.719824 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.719808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:29:37.722083 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.722063    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 03:29:37.725847 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.725826    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c694979d9-trlhz"]
Jul 24 03:29:37.810486 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.810456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzsvg\" (UniqueName: \"kubernetes.io/projected/da79c9d7-9c13-4b0d-8d07-e363700d220f-kube-api-access-kzsvg\") pod \"authorino-6c694979d9-trlhz\" (UID: \"da79c9d7-9c13-4b0d-8d07-e363700d220f\") " pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:29:37.810744 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.810528    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/da79c9d7-9c13-4b0d-8d07-e363700d220f-tls-cert\") pod \"authorino-6c694979d9-trlhz\" (UID: \"da79c9d7-9c13-4b0d-8d07-e363700d220f\") " pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:29:37.877829 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.877773    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zzdp5"
Jul 24 03:29:37.911625 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.911589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/da79c9d7-9c13-4b0d-8d07-e363700d220f-tls-cert\") pod \"authorino-6c694979d9-trlhz\" (UID: \"da79c9d7-9c13-4b0d-8d07-e363700d220f\") " pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:29:37.911801 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.911668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzsvg\" (UniqueName: \"kubernetes.io/projected/da79c9d7-9c13-4b0d-8d07-e363700d220f-kube-api-access-kzsvg\") pod \"authorino-6c694979d9-trlhz\" (UID: \"da79c9d7-9c13-4b0d-8d07-e363700d220f\") " pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:29:37.914171 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.914147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/da79c9d7-9c13-4b0d-8d07-e363700d220f-tls-cert\") pod \"authorino-6c694979d9-trlhz\" (UID: \"da79c9d7-9c13-4b0d-8d07-e363700d220f\") " pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:29:37.919609 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:37.919583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzsvg\" (UniqueName: \"kubernetes.io/projected/da79c9d7-9c13-4b0d-8d07-e363700d220f-kube-api-access-kzsvg\") pod \"authorino-6c694979d9-trlhz\" (UID: \"da79c9d7-9c13-4b0d-8d07-e363700d220f\") " pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:29:38.012069 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.012035    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7gfn\" (UniqueName: \"kubernetes.io/projected/03f87bf6-cde3-4235-9302-fed144e4603a-kube-api-access-q7gfn\") pod \"03f87bf6-cde3-4235-9302-fed144e4603a\" (UID: \"03f87bf6-cde3-4235-9302-fed144e4603a\") "
Jul 24 03:29:38.014277 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.014237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f87bf6-cde3-4235-9302-fed144e4603a-kube-api-access-q7gfn" (OuterVolumeSpecName: "kube-api-access-q7gfn") pod "03f87bf6-cde3-4235-9302-fed144e4603a" (UID: "03f87bf6-cde3-4235-9302-fed144e4603a"). InnerVolumeSpecName "kube-api-access-q7gfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:29:38.030469 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.030451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:29:38.112795 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.112766    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7gfn\" (UniqueName: \"kubernetes.io/projected/03f87bf6-cde3-4235-9302-fed144e4603a-kube-api-access-q7gfn\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:38.156664 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.156585    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c694979d9-trlhz"]
Jul 24 03:29:38.159789 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:29:38.159761    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda79c9d7_9c13_4b0d_8d07_e363700d220f.slice/crio-8d2ffdcfcfba3be8b84110041d691cb6cf804b1fccab1c78390fceca58f6140f WatchSource:0}: Error finding container 8d2ffdcfcfba3be8b84110041d691cb6cf804b1fccab1c78390fceca58f6140f: Status 404 returned error can't find the container with id 8d2ffdcfcfba3be8b84110041d691cb6cf804b1fccab1c78390fceca58f6140f
Jul 24 03:29:38.631916 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.631875    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bf7bdb849-hn584" event={"ID":"d19cb7d9-26c1-4965-a733-b3b360777b1a","Type":"ContainerStarted","Data":"944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256"}
Jul 24 03:29:38.632412 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.631948    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bf7bdb849-hn584" podUID="d19cb7d9-26c1-4965-a733-b3b360777b1a" containerName="authorino" containerID="cri-o://944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256" gracePeriod=30
Jul 24 03:29:38.633415 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.633385    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c694979d9-trlhz" event={"ID":"da79c9d7-9c13-4b0d-8d07-e363700d220f","Type":"ContainerStarted","Data":"8d2ffdcfcfba3be8b84110041d691cb6cf804b1fccab1c78390fceca58f6140f"}
Jul 24 03:29:38.634685 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.634655    2577 generic.go:356] "Generic (PLEG): container finished" podID="03f87bf6-cde3-4235-9302-fed144e4603a" containerID="c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d" exitCode=0
Jul 24 03:29:38.634784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.634709    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-zzdp5"
Jul 24 03:29:38.634784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.634728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-zzdp5" event={"ID":"03f87bf6-cde3-4235-9302-fed144e4603a","Type":"ContainerDied","Data":"c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d"}
Jul 24 03:29:38.634784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.634761    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-zzdp5" event={"ID":"03f87bf6-cde3-4235-9302-fed144e4603a","Type":"ContainerDied","Data":"a18cabcb833ec21c2a8175f4793fd34a1292f2ee9184ab608abc093f58c10ab1"}
Jul 24 03:29:38.634784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.634783    2577 scope.go:117] "RemoveContainer" containerID="c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d"
Jul 24 03:29:38.644414 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.644397    2577 scope.go:117] "RemoveContainer" containerID="c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d"
Jul 24 03:29:38.644651 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:29:38.644630    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d\": container with ID starting with c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d not found: ID does not exist" containerID="c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d"
Jul 24 03:29:38.644735 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.644658    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d"} err="failed to get container status \"c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d\": rpc error: code = NotFound desc = could not find container \"c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d\": container with ID starting with c29854346f79a3882a3e890d02bf44c3cb95271dd78c98bf0de426fbab5f8d3d not found: ID does not exist"
Jul 24 03:29:38.647546 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.647505    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bf7bdb849-hn584" podStartSLOduration=2.24651995 podStartE2EDuration="2.647491017s" totalImagesPullingTime="400.971067ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:29:38.645671739 +0000 UTC m=+745.406580219" watchObservedRunningTime="2026-07-24 03:29:38.647491017 +0000 UTC m=+745.408399498"
Jul 24 03:29:38.688904 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.688875    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zzdp5"]
Jul 24 03:29:38.693101 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.693078    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-zzdp5"]
Jul 24 03:29:38.890586 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:38.890525    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bf7bdb849-hn584"
Jul 24 03:29:39.024688 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.024636    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j8p9\" (UniqueName: \"kubernetes.io/projected/d19cb7d9-26c1-4965-a733-b3b360777b1a-kube-api-access-5j8p9\") pod \"d19cb7d9-26c1-4965-a733-b3b360777b1a\" (UID: \"d19cb7d9-26c1-4965-a733-b3b360777b1a\") "
Jul 24 03:29:39.026924 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.026893    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19cb7d9-26c1-4965-a733-b3b360777b1a-kube-api-access-5j8p9" (OuterVolumeSpecName: "kube-api-access-5j8p9") pod "d19cb7d9-26c1-4965-a733-b3b360777b1a" (UID: "d19cb7d9-26c1-4965-a733-b3b360777b1a"). InnerVolumeSpecName "kube-api-access-5j8p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:29:39.126213 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.126178    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j8p9\" (UniqueName: \"kubernetes.io/projected/d19cb7d9-26c1-4965-a733-b3b360777b1a-kube-api-access-5j8p9\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:39.641996 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.641962    2577 generic.go:356] "Generic (PLEG): container finished" podID="d19cb7d9-26c1-4965-a733-b3b360777b1a" containerID="944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256" exitCode=0
Jul 24 03:29:39.642689 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.642074    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bf7bdb849-hn584"
Jul 24 03:29:39.642689 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.642372    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03f87bf6-cde3-4235-9302-fed144e4603a" path="/var/lib/kubelet/pods/03f87bf6-cde3-4235-9302-fed144e4603a/volumes"
Jul 24 03:29:39.642859 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.642836    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bf7bdb849-hn584" event={"ID":"d19cb7d9-26c1-4965-a733-b3b360777b1a","Type":"ContainerDied","Data":"944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256"}
Jul 24 03:29:39.642912 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.642878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bf7bdb849-hn584" event={"ID":"d19cb7d9-26c1-4965-a733-b3b360777b1a","Type":"ContainerDied","Data":"db3d0e00340385f992c078d1117ce0125ec5b48cc37f1702d12396c83d13964e"}
Jul 24 03:29:39.642912 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.642903    2577 scope.go:117] "RemoveContainer" containerID="944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256"
Jul 24 03:29:39.643963 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.643804    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c694979d9-trlhz" event={"ID":"da79c9d7-9c13-4b0d-8d07-e363700d220f","Type":"ContainerStarted","Data":"4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845"}
Jul 24 03:29:39.652641 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.652624    2577 scope.go:117] "RemoveContainer" containerID="944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256"
Jul 24 03:29:39.652894 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:29:39.652875    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256\": container with ID starting with 944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256 not found: ID does not exist" containerID="944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256"
Jul 24 03:29:39.652968 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.652900    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256"} err="failed to get container status \"944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256\": rpc error: code = NotFound desc = could not find container \"944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256\": container with ID starting with 944c838963d56b8c9753832d1903cc550d5efa545bb329721c224a1ff3aed256 not found: ID does not exist"
Jul 24 03:29:39.662109 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.662067    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c694979d9-trlhz" podStartSLOduration=2.189676397 podStartE2EDuration="2.662054796s" totalImagesPullingTime="472.378399ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:29:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:29:39.660690433 +0000 UTC m=+746.421598916" watchObservedRunningTime="2026-07-24 03:29:39.662054796 +0000 UTC m=+746.422963276"
Jul 24 03:29:39.676642 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.676610    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bf7bdb849-hn584"]
Jul 24 03:29:39.679711 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.679691    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bf7bdb849-hn584"]
Jul 24 03:29:39.689084 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.689063    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rpkpm"]
Jul 24 03:29:39.689250 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.689231    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-rpkpm" podUID="4e99bff8-3f68-4d24-b1cb-a6e7050bfac2" containerName="authorino" containerID="cri-o://da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500" gracePeriod=30
Jul 24 03:29:39.934249 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:39.934225    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rpkpm"
Jul 24 03:29:40.035941 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.035887    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdgzk\" (UniqueName: \"kubernetes.io/projected/4e99bff8-3f68-4d24-b1cb-a6e7050bfac2-kube-api-access-vdgzk\") pod \"4e99bff8-3f68-4d24-b1cb-a6e7050bfac2\" (UID: \"4e99bff8-3f68-4d24-b1cb-a6e7050bfac2\") "
Jul 24 03:29:40.038226 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.038197    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e99bff8-3f68-4d24-b1cb-a6e7050bfac2-kube-api-access-vdgzk" (OuterVolumeSpecName: "kube-api-access-vdgzk") pod "4e99bff8-3f68-4d24-b1cb-a6e7050bfac2" (UID: "4e99bff8-3f68-4d24-b1cb-a6e7050bfac2"). InnerVolumeSpecName "kube-api-access-vdgzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:29:40.136508 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.136471    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdgzk\" (UniqueName: \"kubernetes.io/projected/4e99bff8-3f68-4d24-b1cb-a6e7050bfac2-kube-api-access-vdgzk\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:29:40.652121 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.652086    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e99bff8-3f68-4d24-b1cb-a6e7050bfac2" containerID="da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500" exitCode=0
Jul 24 03:29:40.652629 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.652133    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rpkpm"
Jul 24 03:29:40.652629 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.652171    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rpkpm" event={"ID":"4e99bff8-3f68-4d24-b1cb-a6e7050bfac2","Type":"ContainerDied","Data":"da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500"}
Jul 24 03:29:40.652629 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.652212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rpkpm" event={"ID":"4e99bff8-3f68-4d24-b1cb-a6e7050bfac2","Type":"ContainerDied","Data":"ac3f55b9f26bcc12b37a670a7b66244d7c3097b404ced420b03935c632024b8f"}
Jul 24 03:29:40.652629 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.652230    2577 scope.go:117] "RemoveContainer" containerID="da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500"
Jul 24 03:29:40.661225 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.661210    2577 scope.go:117] "RemoveContainer" containerID="da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500"
Jul 24 03:29:40.661504 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:29:40.661485    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500\": container with ID starting with da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500 not found: ID does not exist" containerID="da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500"
Jul 24 03:29:40.661569 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.661513    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500"} err="failed to get container status \"da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500\": rpc error: code = NotFound desc = could not find container \"da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500\": container with ID starting with da907e7bfc14ad2c406224745e055e527bcfa86532f6f44687b0b6b139a57500 not found: ID does not exist"
Jul 24 03:29:40.671959 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.671940    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rpkpm"]
Jul 24 03:29:40.673715 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:40.673696    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rpkpm"]
Jul 24 03:29:41.641649 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:41.641616    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e99bff8-3f68-4d24-b1cb-a6e7050bfac2" path="/var/lib/kubelet/pods/4e99bff8-3f68-4d24-b1cb-a6e7050bfac2/volumes"
Jul 24 03:29:41.641932 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:29:41.641920    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d19cb7d9-26c1-4965-a733-b3b360777b1a" path="/var/lib/kubelet/pods/d19cb7d9-26c1-4965-a733-b3b360777b1a/volumes"
Jul 24 03:30:20.024591 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:30:20.024556    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:30:42.485579 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:30:42.485496    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:30:52.484460 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:30:52.484423    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:30:59.106482 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:30:59.106436    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:31:08.092378 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:31:08.092342    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:31:32.696689 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:31:32.696653    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:31:45.099238 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:31:45.099203    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:32:45.606345 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606313    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c598c8c48-9xnfn"]
Jul 24 03:32:45.606784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606646    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d19cb7d9-26c1-4965-a733-b3b360777b1a" containerName="authorino"
Jul 24 03:32:45.606784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606657    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19cb7d9-26c1-4965-a733-b3b360777b1a" containerName="authorino"
Jul 24 03:32:45.606784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606669    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e99bff8-3f68-4d24-b1cb-a6e7050bfac2" containerName="authorino"
Jul 24 03:32:45.606784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606674    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e99bff8-3f68-4d24-b1cb-a6e7050bfac2" containerName="authorino"
Jul 24 03:32:45.606784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606685    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f87bf6-cde3-4235-9302-fed144e4603a" containerName="authorino"
Jul 24 03:32:45.606784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606690    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f87bf6-cde3-4235-9302-fed144e4603a" containerName="authorino"
Jul 24 03:32:45.606784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606738    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="03f87bf6-cde3-4235-9302-fed144e4603a" containerName="authorino"
Jul 24 03:32:45.606784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606749    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d19cb7d9-26c1-4965-a733-b3b360777b1a" containerName="authorino"
Jul 24 03:32:45.606784 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.606756    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e99bff8-3f68-4d24-b1cb-a6e7050bfac2" containerName="authorino"
Jul 24 03:32:45.609629 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.609613    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c598c8c48-9xnfn"
Jul 24 03:32:45.617490 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.617465    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c598c8c48-9xnfn"]
Jul 24 03:32:45.667003 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.666969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549-tls-cert\") pod \"authorino-7c598c8c48-9xnfn\" (UID: \"3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549\") " pod="kuadrant-system/authorino-7c598c8c48-9xnfn"
Jul 24 03:32:45.667180 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.667021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qggcm\" (UniqueName: \"kubernetes.io/projected/3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549-kube-api-access-qggcm\") pod \"authorino-7c598c8c48-9xnfn\" (UID: \"3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549\") " pod="kuadrant-system/authorino-7c598c8c48-9xnfn"
Jul 24 03:32:45.768276 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.768242    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qggcm\" (UniqueName: \"kubernetes.io/projected/3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549-kube-api-access-qggcm\") pod \"authorino-7c598c8c48-9xnfn\" (UID: \"3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549\") " pod="kuadrant-system/authorino-7c598c8c48-9xnfn"
Jul 24 03:32:45.768438 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.768353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549-tls-cert\") pod \"authorino-7c598c8c48-9xnfn\" (UID: \"3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549\") " pod="kuadrant-system/authorino-7c598c8c48-9xnfn"
Jul 24 03:32:45.770954 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.770931    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549-tls-cert\") pod \"authorino-7c598c8c48-9xnfn\" (UID: \"3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549\") " pod="kuadrant-system/authorino-7c598c8c48-9xnfn"
Jul 24 03:32:45.776364 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.776341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qggcm\" (UniqueName: \"kubernetes.io/projected/3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549-kube-api-access-qggcm\") pod \"authorino-7c598c8c48-9xnfn\" (UID: \"3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549\") " pod="kuadrant-system/authorino-7c598c8c48-9xnfn"
Jul 24 03:32:45.919111 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:45.919079    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c598c8c48-9xnfn"
Jul 24 03:32:46.077971 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:46.077947    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c598c8c48-9xnfn"]
Jul 24 03:32:46.080689 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:32:46.080662    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3313e4d4_e8e0_43a1_9ce4_4c76e3c3b549.slice/crio-8773303727c43d8dfdaa60db101eacd45957b0409704cc9153c3773f2318a3fe WatchSource:0}: Error finding container 8773303727c43d8dfdaa60db101eacd45957b0409704cc9153c3773f2318a3fe: Status 404 returned error can't find the container with id 8773303727c43d8dfdaa60db101eacd45957b0409704cc9153c3773f2318a3fe
Jul 24 03:32:46.552735 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:46.552700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c598c8c48-9xnfn" event={"ID":"3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549","Type":"ContainerStarted","Data":"8773303727c43d8dfdaa60db101eacd45957b0409704cc9153c3773f2318a3fe"}
Jul 24 03:32:47.561832 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:47.561797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c598c8c48-9xnfn" event={"ID":"3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549","Type":"ContainerStarted","Data":"291027df39b787847f4d49777adeb252c58eb18b496be150209a012df72320fa"}
Jul 24 03:32:47.584161 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:47.584106    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c598c8c48-9xnfn" podStartSLOduration=2.064904235 podStartE2EDuration="2.584088786s" totalImagesPullingTime="519.184551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:32:47.582063525 +0000 UTC m=+934.342971997" watchObservedRunningTime="2026-07-24 03:32:47.584088786 +0000 UTC m=+934.344997298"
Jul 24 03:32:47.654063 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:47.654025    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c694979d9-trlhz"]
Jul 24 03:32:47.654337 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:47.654300    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6c694979d9-trlhz" podUID="da79c9d7-9c13-4b0d-8d07-e363700d220f" containerName="authorino" containerID="cri-o://4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845" gracePeriod=30
Jul 24 03:32:47.919777 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:47.919743    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:32:47.990135 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:47.990088    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzsvg\" (UniqueName: \"kubernetes.io/projected/da79c9d7-9c13-4b0d-8d07-e363700d220f-kube-api-access-kzsvg\") pod \"da79c9d7-9c13-4b0d-8d07-e363700d220f\" (UID: \"da79c9d7-9c13-4b0d-8d07-e363700d220f\") "
Jul 24 03:32:47.990353 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:47.990154    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/da79c9d7-9c13-4b0d-8d07-e363700d220f-tls-cert\") pod \"da79c9d7-9c13-4b0d-8d07-e363700d220f\" (UID: \"da79c9d7-9c13-4b0d-8d07-e363700d220f\") "
Jul 24 03:32:47.992519 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:47.992483    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da79c9d7-9c13-4b0d-8d07-e363700d220f-kube-api-access-kzsvg" (OuterVolumeSpecName: "kube-api-access-kzsvg") pod "da79c9d7-9c13-4b0d-8d07-e363700d220f" (UID: "da79c9d7-9c13-4b0d-8d07-e363700d220f"). InnerVolumeSpecName "kube-api-access-kzsvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:32:48.000726 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.000691    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da79c9d7-9c13-4b0d-8d07-e363700d220f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "da79c9d7-9c13-4b0d-8d07-e363700d220f" (UID: "da79c9d7-9c13-4b0d-8d07-e363700d220f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:32:48.091506 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.091421    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzsvg\" (UniqueName: \"kubernetes.io/projected/da79c9d7-9c13-4b0d-8d07-e363700d220f-kube-api-access-kzsvg\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:32:48.091506 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.091454    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/da79c9d7-9c13-4b0d-8d07-e363700d220f-tls-cert\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:32:48.569087 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.569047    2577 generic.go:356] "Generic (PLEG): container finished" podID="da79c9d7-9c13-4b0d-8d07-e363700d220f" containerID="4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845" exitCode=0
Jul 24 03:32:48.569576 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.569092    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c694979d9-trlhz"
Jul 24 03:32:48.569576 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.569131    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c694979d9-trlhz" event={"ID":"da79c9d7-9c13-4b0d-8d07-e363700d220f","Type":"ContainerDied","Data":"4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845"}
Jul 24 03:32:48.569576 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.569185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c694979d9-trlhz" event={"ID":"da79c9d7-9c13-4b0d-8d07-e363700d220f","Type":"ContainerDied","Data":"8d2ffdcfcfba3be8b84110041d691cb6cf804b1fccab1c78390fceca58f6140f"}
Jul 24 03:32:48.569576 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.569210    2577 scope.go:117] "RemoveContainer" containerID="4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845"
Jul 24 03:32:48.578943 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.578925    2577 scope.go:117] "RemoveContainer" containerID="4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845"
Jul 24 03:32:48.579203 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:32:48.579184    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845\": container with ID starting with 4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845 not found: ID does not exist" containerID="4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845"
Jul 24 03:32:48.579298 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.579215    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845"} err="failed to get container status \"4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845\": rpc error: code = NotFound desc = could not find container \"4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845\": container with ID starting with 4cebe2c8c4ba15f551251a21fc16ad327336bde8123e8ec56492b63a2f23a845 not found: ID does not exist"
Jul 24 03:32:48.590078 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.590053    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c694979d9-trlhz"]
Jul 24 03:32:48.595155 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:48.595127    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c694979d9-trlhz"]
Jul 24 03:32:49.642595 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:32:49.642562    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da79c9d7-9c13-4b0d-8d07-e363700d220f" path="/var/lib/kubelet/pods/da79c9d7-9c13-4b0d-8d07-e363700d220f/volumes"
Jul 24 03:33:15.390096 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:33:15.390061    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:33:25.588619 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:33:25.588589    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:33:36.188863 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:33:36.188766    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:35:17.097780 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:35:17.097697    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:35:30.791132 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:35:30.791096    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:36:04.796998 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:36:04.796961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:36:12.588415 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:36:12.588381    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:36:23.489921 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:36:23.489880    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:36:30.800031 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:36:30.799990    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:36:42.201804 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:36:42.201727    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:36:52.129173 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:36:52.129131    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:37:03.091869 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:37:03.091837    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:37:12.993172 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:37:12.993138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:37:38.885081 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:37:38.885050    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:37:48.281501 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:37:48.281462    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:38:10.899166 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:38:10.899080    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:38:21.097205 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:38:21.097172    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:38:38.988825 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:38:38.988778    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:38:48.695639 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:38:48.695594    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:38:58.191155 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:38:58.191119    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:39:07.392788 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:39:07.392747    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:39:16.294379 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:39:16.294341    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:41:06.090588 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:41:06.090482    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:41:15.290246 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:41:15.290215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:41:27.899457 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:41:27.899424    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:41:39.685687 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:41:39.685647    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:41:50.686966 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:41:50.686932    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:42:01.298977 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:42:01.298945    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:42:10.691848 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:42:10.691807    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:42:21.595496 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:42:21.595465    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:42:33.188082 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:42:33.188048    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:42:42.299273 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:42:42.299234    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:42:53.988395 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:42:53.988360    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:43:16.199343 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:16.199295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:43:26.388067 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:26.388033    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:43:31.349512 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.349479    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"]
Jul 24 03:43:31.349849 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.349836    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da79c9d7-9c13-4b0d-8d07-e363700d220f" containerName="authorino"
Jul 24 03:43:31.349849 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.349847    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="da79c9d7-9c13-4b0d-8d07-e363700d220f" containerName="authorino"
Jul 24 03:43:31.349915 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.349907    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="da79c9d7-9c13-4b0d-8d07-e363700d220f" containerName="authorino"
Jul 24 03:43:31.352898 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.352880    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.355456 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.355435    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p7qhl\""
Jul 24 03:43:31.355456 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.355446    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 03:43:31.356370 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.356356    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 03:43:31.356439 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.356423    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-78465f79-kserve-self-signed-certs\""
Jul 24 03:43:31.370756 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.370736    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"]
Jul 24 03:43:31.372930 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.372910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-dshm\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.373018 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.372945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tls-certs\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.373018 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.372964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-model-cache\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.373126 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.373052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-home\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.373126 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.373092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tmp-dir\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.373218 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.373133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgr2w\" (UniqueName: \"kubernetes.io/projected/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-kube-api-access-hgr2w\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.473462 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.473438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-dshm\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.473561 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.473469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tls-certs\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.473561 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.473488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-model-cache\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.473561 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.473514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-home\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.473561 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.473531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tmp-dir\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.473561 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.473551    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgr2w\" (UniqueName: \"kubernetes.io/projected/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-kube-api-access-hgr2w\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.473899 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.473874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tmp-dir\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.473976 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.473921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-home\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.473976 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.473934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-model-cache\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.475757 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.475734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-dshm\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.475914 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.475896    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tls-certs\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.481829 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.481809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgr2w\" (UniqueName: \"kubernetes.io/projected/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-kube-api-access-hgr2w\") pod \"e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") " pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.661967 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.661944    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:31.787736 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.787707    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"]
Jul 24 03:43:31.789810 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:43:31.789785    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3d6a2a6_e1a7_411d_8805_8e458724a3ae.slice/crio-cad312207ea2150762d1b0320b025c059258c572afbdcce40b56c3baddff64eb WatchSource:0}: Error finding container cad312207ea2150762d1b0320b025c059258c572afbdcce40b56c3baddff64eb: Status 404 returned error can't find the container with id cad312207ea2150762d1b0320b025c059258c572afbdcce40b56c3baddff64eb
Jul 24 03:43:31.791703 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:31.791685    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 03:43:32.691699 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:32.691660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6" event={"ID":"e3d6a2a6-e1a7-411d-8805-8e458724a3ae","Type":"ContainerStarted","Data":"cad312207ea2150762d1b0320b025c059258c572afbdcce40b56c3baddff64eb"}
Jul 24 03:43:33.696959 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:33.696930    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6" event={"ID":"e3d6a2a6-e1a7-411d-8805-8e458724a3ae","Type":"ContainerStarted","Data":"4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496"}
Jul 24 03:43:33.713133 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:33.713086    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6" podStartSLOduration=1.654521302 podStartE2EDuration="2.713071278s" totalImagesPullingTime="1.058549976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:43:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:43:33.711699225 +0000 UTC m=+1580.472607705" watchObservedRunningTime="2026-07-24 03:43:33.713071278 +0000 UTC m=+1580.473979735"
Jul 24 03:43:41.662662 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.662641    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:41.663038 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.662683    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:41.675105 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.675073    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:41.745754 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.745732    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:43:41.863071 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.863046    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"]
Jul 24 03:43:41.866775 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.866758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:41.871525 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.871504    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-78465f79-kserve-self-signed-certs\""
Jul 24 03:43:41.888441 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.888418    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"]
Jul 24 03:43:41.950364 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.950309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3cbb430-6435-4a07-8704-7aef2be1bf02-tls-certs\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:41.950364 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.950341    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-model-cache\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:41.950488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.950390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-home\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:41.950488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.950414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-dshm\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:41.950488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.950440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czncq\" (UniqueName: \"kubernetes.io/projected/e3cbb430-6435-4a07-8704-7aef2be1bf02-kube-api-access-czncq\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:41.950799 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:41.950497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-tmp-dir\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.051586 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.051563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-home\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.051679 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.051590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-dshm\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.051679 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.051612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czncq\" (UniqueName: \"kubernetes.io/projected/e3cbb430-6435-4a07-8704-7aef2be1bf02-kube-api-access-czncq\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.051760 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.051738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-tmp-dir\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.051799 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.051790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3cbb430-6435-4a07-8704-7aef2be1bf02-tls-certs\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.051844 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.051811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-model-cache\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.051963 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.051942    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-home\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.052081 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.052060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-tmp-dir\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.052147 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.052104    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-model-cache\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.053831 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.053813    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-dshm\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.054139 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.054120    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3cbb430-6435-4a07-8704-7aef2be1bf02-tls-certs\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.061397 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.061375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czncq\" (UniqueName: \"kubernetes.io/projected/e3cbb430-6435-4a07-8704-7aef2be1bf02-kube-api-access-czncq\") pod \"e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") " pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.174943 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.174925    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:42.301035 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.300911    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"]
Jul 24 03:43:42.303615 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:43:42.303587    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3cbb430_6435_4a07_8704_7aef2be1bf02.slice/crio-670ba6a67a1b1f0128defbfb97a911d4cf4ddec34123b79c8541274de3bd2ef0 WatchSource:0}: Error finding container 670ba6a67a1b1f0128defbfb97a911d4cf4ddec34123b79c8541274de3bd2ef0: Status 404 returned error can't find the container with id 670ba6a67a1b1f0128defbfb97a911d4cf4ddec34123b79c8541274de3bd2ef0
Jul 24 03:43:42.741737 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.741699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq" event={"ID":"e3cbb430-6435-4a07-8704-7aef2be1bf02","Type":"ContainerStarted","Data":"5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27"}
Jul 24 03:43:42.742094 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.741747    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq" event={"ID":"e3cbb430-6435-4a07-8704-7aef2be1bf02","Type":"ContainerStarted","Data":"670ba6a67a1b1f0128defbfb97a911d4cf4ddec34123b79c8541274de3bd2ef0"}
Jul 24 03:43:42.761153 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:42.761107    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq" podStartSLOduration=1.761093416 podStartE2EDuration="1.761093416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:43:42.75965189 +0000 UTC m=+1589.520560368" watchObservedRunningTime="2026-07-24 03:43:42.761093416 +0000 UTC m=+1589.522001894"
Jul 24 03:43:44.592071 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:44.592036    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:43:52.175817 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:52.175782    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:52.176211 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:52.175831    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:52.188091 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:52.188062    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:43:52.798696 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:43:52.798673    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:44:07.372688 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:07.372617    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"]
Jul 24 03:44:08.439100 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:08.439062    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"]
Jul 24 03:44:11.202936 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:11.202899    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:44:22.391424 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.391357    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6" podUID="e3d6a2a6-e1a7-411d-8805-8e458724a3ae" containerName="main" containerID="cri-o://4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496" gracePeriod=45
Jul 24 03:44:22.628006 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.627978    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:44:22.779102 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779053    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-dshm\") pod \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") "
Jul 24 03:44:22.779102 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779108    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tmp-dir\") pod \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") "
Jul 24 03:44:22.779391 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779145    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-home\") pod \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") "
Jul 24 03:44:22.779391 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779211    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgr2w\" (UniqueName: \"kubernetes.io/projected/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-kube-api-access-hgr2w\") pod \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") "
Jul 24 03:44:22.779391 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779256    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tls-certs\") pod \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") "
Jul 24 03:44:22.779391 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779313    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-model-cache\") pod \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\" (UID: \"e3d6a2a6-e1a7-411d-8805-8e458724a3ae\") "
Jul 24 03:44:22.779595 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779441    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-home" (OuterVolumeSpecName: "home") pod "e3d6a2a6-e1a7-411d-8805-8e458724a3ae" (UID: "e3d6a2a6-e1a7-411d-8805-8e458724a3ae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:44:22.779595 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779450    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3d6a2a6-e1a7-411d-8805-8e458724a3ae" (UID: "e3d6a2a6-e1a7-411d-8805-8e458724a3ae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:44:22.779706 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779609    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3d6a2a6-e1a7-411d-8805-8e458724a3ae" (UID: "e3d6a2a6-e1a7-411d-8805-8e458724a3ae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:44:22.779915 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779777    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:22.779915 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779802    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:22.779915 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.779814    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:22.781476 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.781459    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-dshm" (OuterVolumeSpecName: "dshm") pod "e3d6a2a6-e1a7-411d-8805-8e458724a3ae" (UID: "e3d6a2a6-e1a7-411d-8805-8e458724a3ae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:44:22.781923 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.781904    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3d6a2a6-e1a7-411d-8805-8e458724a3ae" (UID: "e3d6a2a6-e1a7-411d-8805-8e458724a3ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:44:22.781971 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.781940    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-kube-api-access-hgr2w" (OuterVolumeSpecName: "kube-api-access-hgr2w") pod "e3d6a2a6-e1a7-411d-8805-8e458724a3ae" (UID: "e3d6a2a6-e1a7-411d-8805-8e458724a3ae"). InnerVolumeSpecName "kube-api-access-hgr2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:44:22.880596 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.880567    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:22.880596 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.880591    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgr2w\" (UniqueName: \"kubernetes.io/projected/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-kube-api-access-hgr2w\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:22.880596 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.880601    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d6a2a6-e1a7-411d-8805-8e458724a3ae-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:22.935231 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.935200    2577 generic.go:356] "Generic (PLEG): container finished" podID="e3d6a2a6-e1a7-411d-8805-8e458724a3ae" containerID="4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496" exitCode=0
Jul 24 03:44:22.935379 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.935234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6" event={"ID":"e3d6a2a6-e1a7-411d-8805-8e458724a3ae","Type":"ContainerDied","Data":"4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496"}
Jul 24 03:44:22.935379 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.935292    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6" event={"ID":"e3d6a2a6-e1a7-411d-8805-8e458724a3ae","Type":"ContainerDied","Data":"cad312207ea2150762d1b0320b025c059258c572afbdcce40b56c3baddff64eb"}
Jul 24 03:44:22.935379 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.935309    2577 scope.go:117] "RemoveContainer" containerID="4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496"
Jul 24 03:44:22.935379 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.935243    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"
Jul 24 03:44:22.944496 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.944481    2577 scope.go:117] "RemoveContainer" containerID="4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496"
Jul 24 03:44:22.944732 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:44:22.944712    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496\": container with ID starting with 4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496 not found: ID does not exist" containerID="4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496"
Jul 24 03:44:22.944788 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.944740    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496"} err="failed to get container status \"4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496\": rpc error: code = NotFound desc = could not find container \"4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496\": container with ID starting with 4e428afaeda7bd8158cbc5a1f725593dfd35cd449b868e440d8c9b689fd19496 not found: ID does not exist"
Jul 24 03:44:22.961671 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.961653    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"]
Jul 24 03:44:22.965446 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:22.965424    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-78465f79-kserve-7b675c6655-lzbr6"]
Jul 24 03:44:23.457923 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.457880    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq" podUID="e3cbb430-6435-4a07-8704-7aef2be1bf02" containerName="main" containerID="cri-o://5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27" gracePeriod=45
Jul 24 03:44:23.642966 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.642928    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3d6a2a6-e1a7-411d-8805-8e458724a3ae" path="/var/lib/kubelet/pods/e3d6a2a6-e1a7-411d-8805-8e458724a3ae/volumes"
Jul 24 03:44:23.704194 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.704167    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:44:23.788131 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.788075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-dshm\") pod \"e3cbb430-6435-4a07-8704-7aef2be1bf02\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") "
Jul 24 03:44:23.788131 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.788128    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3cbb430-6435-4a07-8704-7aef2be1bf02-tls-certs\") pod \"e3cbb430-6435-4a07-8704-7aef2be1bf02\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") "
Jul 24 03:44:23.788292 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.788159    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-model-cache\") pod \"e3cbb430-6435-4a07-8704-7aef2be1bf02\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") "
Jul 24 03:44:23.788292 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.788174    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-tmp-dir\") pod \"e3cbb430-6435-4a07-8704-7aef2be1bf02\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") "
Jul 24 03:44:23.788292 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.788210    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czncq\" (UniqueName: \"kubernetes.io/projected/e3cbb430-6435-4a07-8704-7aef2be1bf02-kube-api-access-czncq\") pod \"e3cbb430-6435-4a07-8704-7aef2be1bf02\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") "
Jul 24 03:44:23.788292 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.788235    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-home\") pod \"e3cbb430-6435-4a07-8704-7aef2be1bf02\" (UID: \"e3cbb430-6435-4a07-8704-7aef2be1bf02\") "
Jul 24 03:44:23.788507 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.788472    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3cbb430-6435-4a07-8704-7aef2be1bf02" (UID: "e3cbb430-6435-4a07-8704-7aef2be1bf02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:44:23.788507 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.788490    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3cbb430-6435-4a07-8704-7aef2be1bf02" (UID: "e3cbb430-6435-4a07-8704-7aef2be1bf02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:44:23.788638 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.788622    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-home" (OuterVolumeSpecName: "home") pod "e3cbb430-6435-4a07-8704-7aef2be1bf02" (UID: "e3cbb430-6435-4a07-8704-7aef2be1bf02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:44:23.790153 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.790135    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-dshm" (OuterVolumeSpecName: "dshm") pod "e3cbb430-6435-4a07-8704-7aef2be1bf02" (UID: "e3cbb430-6435-4a07-8704-7aef2be1bf02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:44:23.790333 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.790313    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3cbb430-6435-4a07-8704-7aef2be1bf02-kube-api-access-czncq" (OuterVolumeSpecName: "kube-api-access-czncq") pod "e3cbb430-6435-4a07-8704-7aef2be1bf02" (UID: "e3cbb430-6435-4a07-8704-7aef2be1bf02"). InnerVolumeSpecName "kube-api-access-czncq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:44:23.790485 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.790467    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cbb430-6435-4a07-8704-7aef2be1bf02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3cbb430-6435-4a07-8704-7aef2be1bf02" (UID: "e3cbb430-6435-4a07-8704-7aef2be1bf02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:44:23.888903 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.888879    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:23.888903 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.888900    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3cbb430-6435-4a07-8704-7aef2be1bf02-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:23.889024 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.888910    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:23.889024 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.888918    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:23.889024 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.888927    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czncq\" (UniqueName: \"kubernetes.io/projected/e3cbb430-6435-4a07-8704-7aef2be1bf02-kube-api-access-czncq\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:23.889024 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.888935    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb430-6435-4a07-8704-7aef2be1bf02-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:44:23.940742 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.940710    2577 generic.go:356] "Generic (PLEG): container finished" podID="e3cbb430-6435-4a07-8704-7aef2be1bf02" containerID="5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27" exitCode=0
Jul 24 03:44:23.940843 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.940757    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"
Jul 24 03:44:23.940843 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.940755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq" event={"ID":"e3cbb430-6435-4a07-8704-7aef2be1bf02","Type":"ContainerDied","Data":"5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27"}
Jul 24 03:44:23.940950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.940865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq" event={"ID":"e3cbb430-6435-4a07-8704-7aef2be1bf02","Type":"ContainerDied","Data":"670ba6a67a1b1f0128defbfb97a911d4cf4ddec34123b79c8541274de3bd2ef0"}
Jul 24 03:44:23.940950 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.940887    2577 scope.go:117] "RemoveContainer" containerID="5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27"
Jul 24 03:44:23.950018 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.950004    2577 scope.go:117] "RemoveContainer" containerID="5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27"
Jul 24 03:44:23.950280 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:44:23.950247    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27\": container with ID starting with 5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27 not found: ID does not exist" containerID="5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27"
Jul 24 03:44:23.950414 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.950387    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27"} err="failed to get container status \"5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27\": rpc error: code = NotFound desc = could not find container \"5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27\": container with ID starting with 5b780743ac5f3b8632863a0b70f91a516b774f0054754996d444e2beb4aa4e27 not found: ID does not exist"
Jul 24 03:44:23.962572 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.962541    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"]
Jul 24 03:44:23.966645 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:23.966622    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-78465f79-kserve-7f5655d4d7-q5mrq"]
Jul 24 03:44:24.901435 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:24.901395    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:44:25.641935 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:25.641905    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3cbb430-6435-4a07-8704-7aef2be1bf02" path="/var/lib/kubelet/pods/e3cbb430-6435-4a07-8704-7aef2be1bf02/volumes"
Jul 24 03:44:31.899343 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:31.899314    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:44:46.799934 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:46.799903    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:44:56.486895 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:44:56.486862    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:45:06.389650 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:45:06.389607    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:45:14.683132 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:45:14.683091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:45:23.720052 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:45:23.720009    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:45:33.338936 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:45:33.338899    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:45:46.795224 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:45:46.795186    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:45:56.407079 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:45:56.407045    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:46:06.488860 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:46:06.488827    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:46:15.892382 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:46:15.892336    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:46:27.994192 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:46:27.994155    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:46:37.392494 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:46:37.392459    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:46:49.501036 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:46:49.500998    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:47:00.898428 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:47:00.898396    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:47:14.098113 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:47:14.098076    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:47:27.393132 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:47:27.393095    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:47:40.100150 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:47:40.100114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:47:56.894370 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:47:56.894323    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:48:28.190922 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:48:28.190889    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:48:39.644700 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:48:39.644619    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:49:12.098604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:49:12.098566    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:49:23.389949 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:49:23.389915    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:49:55.691750 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:49:55.691715    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:50:06.693867 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:50:06.693785    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:50:39.190571 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:50:39.190533    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:52:06.093157 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:52:06.093120    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:52:15.690112 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:52:15.690074    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:52:28.797107 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:52:28.797069    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:52:48.495474 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:52:48.495436    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:53:26.288191 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:26.288093    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:53:42.299649 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.299610    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"]
Jul 24 03:53:42.300016 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.299913    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3d6a2a6-e1a7-411d-8805-8e458724a3ae" containerName="main"
Jul 24 03:53:42.300016 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.299923    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3d6a2a6-e1a7-411d-8805-8e458724a3ae" containerName="main"
Jul 24 03:53:42.300016 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.299934    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cbb430-6435-4a07-8704-7aef2be1bf02" containerName="main"
Jul 24 03:53:42.300016 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.299941    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cbb430-6435-4a07-8704-7aef2be1bf02" containerName="main"
Jul 24 03:53:42.300016 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.300000    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3d6a2a6-e1a7-411d-8805-8e458724a3ae" containerName="main"
Jul 24 03:53:42.300016 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.300007    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cbb430-6435-4a07-8704-7aef2be1bf02" containerName="main"
Jul 24 03:53:42.302984 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.302967    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.306076 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.306045    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e2438b97\"/\"e2e-lifecycle-model-e2438b97-kserve-self-signed-certs\""
Jul 24 03:53:42.307017 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.306995    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e2438b97\"/\"kube-root-ca.crt\""
Jul 24 03:53:42.307017 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.307013    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e2438b97\"/\"openshift-service-ca.crt\""
Jul 24 03:53:42.307196 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.307002    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e2438b97\"/\"default-dockercfg-lbf9x\""
Jul 24 03:53:42.323371 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.323345    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"]
Jul 24 03:53:42.473725 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.473683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-home\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.473725 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.473731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd6a2-7753-46ac-a203-bbc479306d5d-tls-certs\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.473980 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.473752    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwj4l\" (UniqueName: \"kubernetes.io/projected/98dbd6a2-7753-46ac-a203-bbc479306d5d-kube-api-access-wwj4l\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.473980 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.473830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-dshm\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.473980 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.473906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-model-cache\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.474088 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.474007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-tmp-dir\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.574895 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.574801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-tmp-dir\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.574895 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.574840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-home\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.574895 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.574870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd6a2-7753-46ac-a203-bbc479306d5d-tls-certs\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.574895 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.574885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwj4l\" (UniqueName: \"kubernetes.io/projected/98dbd6a2-7753-46ac-a203-bbc479306d5d-kube-api-access-wwj4l\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.575252 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.574913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-dshm\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.575252 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.574989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-model-cache\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.575252 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.575239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-tmp-dir\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.575425 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.575321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-home\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.575425 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.575418    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-model-cache\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.577395 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.577369    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-dshm\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.577552 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.577535    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd6a2-7753-46ac-a203-bbc479306d5d-tls-certs\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.582861 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.582842    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwj4l\" (UniqueName: \"kubernetes.io/projected/98dbd6a2-7753-46ac-a203-bbc479306d5d-kube-api-access-wwj4l\") pod \"e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") " pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.611930 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.611890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:42.745802 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:42.745776    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"]
Jul 24 03:53:42.747918 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:53:42.747888    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98dbd6a2_7753_46ac_a203_bbc479306d5d.slice/crio-86088cad46ecc7b53d005791dd20545fdf411f0f7d556a2666722f054f7668cb WatchSource:0}: Error finding container 86088cad46ecc7b53d005791dd20545fdf411f0f7d556a2666722f054f7668cb: Status 404 returned error can't find the container with id 86088cad46ecc7b53d005791dd20545fdf411f0f7d556a2666722f054f7668cb
Jul 24 03:53:43.648790 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:43.648754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd" event={"ID":"98dbd6a2-7753-46ac-a203-bbc479306d5d","Type":"ContainerStarted","Data":"f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920"}
Jul 24 03:53:43.648790 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:43.648792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd" event={"ID":"98dbd6a2-7753-46ac-a203-bbc479306d5d","Type":"ContainerStarted","Data":"86088cad46ecc7b53d005791dd20545fdf411f0f7d556a2666722f054f7668cb"}
Jul 24 03:53:43.666937 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:43.666881    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd" podStartSLOduration=1.666867361 podStartE2EDuration="1.666867361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:53:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:53:43.665067962 +0000 UTC m=+2190.425976443" watchObservedRunningTime="2026-07-24 03:53:43.666867361 +0000 UTC m=+2190.427775839"
Jul 24 03:53:45.685368 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:45.685332    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:53:52.612253 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:52.612218    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:52.612689 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:52.612305    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:52.624858 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:52.624807    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:52.705382 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:52.705344    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:53:58.992471 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:53:58.992435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:54:17.594134 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:17.594098    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:54:26.484381 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:26.484337    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:54:31.522022 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:31.521983    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"]
Jul 24 03:54:46.541174 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.541063    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd" podUID="98dbd6a2-7753-46ac-a203-bbc479306d5d" containerName="main" containerID="cri-o://f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920" gracePeriod=45
Jul 24 03:54:46.784031 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.784000    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:54:46.828945 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.828841    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-model-cache\") pod \"98dbd6a2-7753-46ac-a203-bbc479306d5d\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") "
Jul 24 03:54:46.828945 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.828936    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-home\") pod \"98dbd6a2-7753-46ac-a203-bbc479306d5d\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") "
Jul 24 03:54:46.829177 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.828967    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-dshm\") pod \"98dbd6a2-7753-46ac-a203-bbc479306d5d\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") "
Jul 24 03:54:46.829177 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.828984    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-tmp-dir\") pod \"98dbd6a2-7753-46ac-a203-bbc479306d5d\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") "
Jul 24 03:54:46.829177 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.829015    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd6a2-7753-46ac-a203-bbc479306d5d-tls-certs\") pod \"98dbd6a2-7753-46ac-a203-bbc479306d5d\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") "
Jul 24 03:54:46.829177 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.829079    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwj4l\" (UniqueName: \"kubernetes.io/projected/98dbd6a2-7753-46ac-a203-bbc479306d5d-kube-api-access-wwj4l\") pod \"98dbd6a2-7753-46ac-a203-bbc479306d5d\" (UID: \"98dbd6a2-7753-46ac-a203-bbc479306d5d\") "
Jul 24 03:54:46.829177 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.829123    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-model-cache" (OuterVolumeSpecName: "model-cache") pod "98dbd6a2-7753-46ac-a203-bbc479306d5d" (UID: "98dbd6a2-7753-46ac-a203-bbc479306d5d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:54:46.829462 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.829249    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98dbd6a2-7753-46ac-a203-bbc479306d5d" (UID: "98dbd6a2-7753-46ac-a203-bbc479306d5d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:54:46.829462 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.829289    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-home" (OuterVolumeSpecName: "home") pod "98dbd6a2-7753-46ac-a203-bbc479306d5d" (UID: "98dbd6a2-7753-46ac-a203-bbc479306d5d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:54:46.829533 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.829464    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:54:46.829533 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.829482    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:54:46.829533 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.829496    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:54:46.831417 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.831390    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98dbd6a2-7753-46ac-a203-bbc479306d5d-kube-api-access-wwj4l" (OuterVolumeSpecName: "kube-api-access-wwj4l") pod "98dbd6a2-7753-46ac-a203-bbc479306d5d" (UID: "98dbd6a2-7753-46ac-a203-bbc479306d5d"). InnerVolumeSpecName "kube-api-access-wwj4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:54:46.831619 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.831598    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-dshm" (OuterVolumeSpecName: "dshm") pod "98dbd6a2-7753-46ac-a203-bbc479306d5d" (UID: "98dbd6a2-7753-46ac-a203-bbc479306d5d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:54:46.831716 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.831698    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98dbd6a2-7753-46ac-a203-bbc479306d5d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98dbd6a2-7753-46ac-a203-bbc479306d5d" (UID: "98dbd6a2-7753-46ac-a203-bbc479306d5d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:54:46.930729 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.930693    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd6a2-7753-46ac-a203-bbc479306d5d-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:54:46.930729 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.930722    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd6a2-7753-46ac-a203-bbc479306d5d-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:54:46.930729 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.930731    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwj4l\" (UniqueName: \"kubernetes.io/projected/98dbd6a2-7753-46ac-a203-bbc479306d5d-kube-api-access-wwj4l\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:54:46.956632 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.956587    2577 generic.go:356] "Generic (PLEG): container finished" podID="98dbd6a2-7753-46ac-a203-bbc479306d5d" containerID="f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920" exitCode=0
Jul 24 03:54:46.956632 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.956623    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"
Jul 24 03:54:46.956833 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.956654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd" event={"ID":"98dbd6a2-7753-46ac-a203-bbc479306d5d","Type":"ContainerDied","Data":"f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920"}
Jul 24 03:54:46.956833 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.956699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd" event={"ID":"98dbd6a2-7753-46ac-a203-bbc479306d5d","Type":"ContainerDied","Data":"86088cad46ecc7b53d005791dd20545fdf411f0f7d556a2666722f054f7668cb"}
Jul 24 03:54:46.956833 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.956723    2577 scope.go:117] "RemoveContainer" containerID="f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920"
Jul 24 03:54:46.965708 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.965692    2577 scope.go:117] "RemoveContainer" containerID="f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920"
Jul 24 03:54:46.965985 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:54:46.965964    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920\": container with ID starting with f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920 not found: ID does not exist" containerID="f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920"
Jul 24 03:54:46.966039 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.965993    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920"} err="failed to get container status \"f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920\": rpc error: code = NotFound desc = could not find container \"f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920\": container with ID starting with f7898591d992ba0ad1487e9d36648be96e22e4114526ba05dea1386c1d282920 not found: ID does not exist"
Jul 24 03:54:46.976455 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.976429    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"]
Jul 24 03:54:46.979896 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:46.979876    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e2438b97/e2e-lifecycle-model-e2438b97-kserve-646ff575d7-5cggd"]
Jul 24 03:54:47.641827 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:54:47.641795    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98dbd6a2-7753-46ac-a203-bbc479306d5d" path="/var/lib/kubelet/pods/98dbd6a2-7753-46ac-a203-bbc479306d5d/volumes"
Jul 24 03:55:24.892015 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:55:24.891973    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:55:34.886489 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:55:34.886452    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:56:25.894422 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:25.894338    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:56:28.361207 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.361172    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"]
Jul 24 03:56:28.361609 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.361538    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98dbd6a2-7753-46ac-a203-bbc479306d5d" containerName="main"
Jul 24 03:56:28.361609 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.361551    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="98dbd6a2-7753-46ac-a203-bbc479306d5d" containerName="main"
Jul 24 03:56:28.361719 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.361611    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="98dbd6a2-7753-46ac-a203-bbc479306d5d" containerName="main"
Jul 24 03:56:28.364655 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.364635    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:28.367235 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.367211    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-58nst\""
Jul 24 03:56:28.367375 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.367219    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b0d347-serving-cert\""
Jul 24 03:56:28.367375 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.367219    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 03:56:28.368102 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.368080    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:56:28.374701 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.374676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"]
Jul 24 03:56:28.404099 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.404061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-485lh\" (UniqueName: \"kubernetes.io/projected/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-kube-api-access-485lh\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:28.404297 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.404116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-maas-api-tls\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:28.404297 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.404136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:28.504734 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.504699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-maas-api-tls\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:28.504734 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.504738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:28.504973 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.504787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-485lh\" (UniqueName: \"kubernetes.io/projected/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-kube-api-access-485lh\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:28.504973 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:56:28.504879    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-serving-cert: secret "maas-api-e2e-inf-b-b0d347-serving-cert" not found
Jul 24 03:56:28.504973 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:56:28.504966    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-maas-api-tls podName:bd272f8e-34f1-47c9-9913-ae1819c3fb1c nodeName:}" failed. No retries permitted until 2026-07-24 03:56:29.004945365 +0000 UTC m=+2355.765853826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-maas-api-tls") pod "maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47" (UID: "bd272f8e-34f1-47c9-9913-ae1819c3fb1c") : secret "maas-api-e2e-inf-b-b0d347-serving-cert" not found
Jul 24 03:56:28.505533 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.505516    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:28.514779 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:28.514752    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-485lh\" (UniqueName: \"kubernetes.io/projected/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-kube-api-access-485lh\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:29.009880 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:29.009824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-maas-api-tls\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:29.012387 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:29.012366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-maas-api-tls\") pod \"maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:29.276047 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:29.275965    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:29.407138 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:29.407113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"]
Jul 24 03:56:29.409014 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:56:29.408985    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd272f8e_34f1_47c9_9913_ae1819c3fb1c.slice/crio-919e57c992522e1e9b1d09e33bf0eff99aaa6ca4c0bab1506635243cd966a04a WatchSource:0}: Error finding container 919e57c992522e1e9b1d09e33bf0eff99aaa6ca4c0bab1506635243cd966a04a: Status 404 returned error can't find the container with id 919e57c992522e1e9b1d09e33bf0eff99aaa6ca4c0bab1506635243cd966a04a
Jul 24 03:56:29.410618 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:29.410600    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 03:56:29.448782 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:29.448744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47" event={"ID":"bd272f8e-34f1-47c9-9913-ae1819c3fb1c","Type":"ContainerStarted","Data":"919e57c992522e1e9b1d09e33bf0eff99aaa6ca4c0bab1506635243cd966a04a"}
Jul 24 03:56:32.466921 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:32.466884    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47" event={"ID":"bd272f8e-34f1-47c9-9913-ae1819c3fb1c","Type":"ContainerStarted","Data":"965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132"}
Jul 24 03:56:32.467550 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:32.466971    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:38.478891 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:38.478859    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:56:38.494605 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:38.494552    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47" podStartSLOduration=8.201299798 podStartE2EDuration="10.494537968s" totalImagesPullingTime="2.29323817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:56:32.486159026 +0000 UTC m=+2359.247067505" watchObservedRunningTime="2026-07-24 03:56:38.494537968 +0000 UTC m=+2365.255446447"
Jul 24 03:56:38.498473 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:38.498446    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:56:48.242273 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.242226    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"]
Jul 24 03:56:48.245738 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.245720    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.248828 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.248798    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-93dff3\"/\"test-model-93dff3-kserve-self-signed-certs\""
Jul 24 03:56:48.249783 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.249764    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-93dff3\"/\"default-dockercfg-g6h64\""
Jul 24 03:56:48.249886 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.249785    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-93dff3\"/\"openshift-service-ca.crt\""
Jul 24 03:56:48.249886 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.249810    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-93dff3\"/\"kube-root-ca.crt\""
Jul 24 03:56:48.255639 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.255614    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"]
Jul 24 03:56:48.375771 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.375720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c321117-b3f8-481e-a03a-cb81daf1986f-tls-certs\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.375951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.375789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wv8r\" (UniqueName: \"kubernetes.io/projected/0c321117-b3f8-481e-a03a-cb81daf1986f-kube-api-access-2wv8r\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.375951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.375815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-home\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.375951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.375847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-tmp-dir\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.375951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.375897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-model-cache\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.375951 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.375936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-dshm\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.476416 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.476382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wv8r\" (UniqueName: \"kubernetes.io/projected/0c321117-b3f8-481e-a03a-cb81daf1986f-kube-api-access-2wv8r\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.476583 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.476425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-home\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.476583 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.476462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-tmp-dir\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.476583 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.476489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-model-cache\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.476583 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.476515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-dshm\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.476583 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.476553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c321117-b3f8-481e-a03a-cb81daf1986f-tls-certs\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.477071 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.477044    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-model-cache\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.477397 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.477134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-tmp-dir\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.477397 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.477157    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-home\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.479007 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.478980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-dshm\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.479253 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.479223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c321117-b3f8-481e-a03a-cb81daf1986f-tls-certs\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.486091 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.486050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wv8r\" (UniqueName: \"kubernetes.io/projected/0c321117-b3f8-481e-a03a-cb81daf1986f-kube-api-access-2wv8r\") pod \"test-model-93dff3-kserve-57445449c9-7wvkr\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") " pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.555057 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.554970    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:48.691015 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:48.690988    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"]
Jul 24 03:56:48.692773 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:56:48.692745    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c321117_b3f8_481e_a03a_cb81daf1986f.slice/crio-5d7394e6c59d702774a1b41cc047030688ef9c9ff206c046547dd31b41f8ee50 WatchSource:0}: Error finding container 5d7394e6c59d702774a1b41cc047030688ef9c9ff206c046547dd31b41f8ee50: Status 404 returned error can't find the container with id 5d7394e6c59d702774a1b41cc047030688ef9c9ff206c046547dd31b41f8ee50
Jul 24 03:56:49.555731 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:49.555689    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr" event={"ID":"0c321117-b3f8-481e-a03a-cb81daf1986f","Type":"ContainerStarted","Data":"7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7"}
Jul 24 03:56:49.556113 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:49.555741    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr" event={"ID":"0c321117-b3f8-481e-a03a-cb81daf1986f","Type":"ContainerStarted","Data":"5d7394e6c59d702774a1b41cc047030688ef9c9ff206c046547dd31b41f8ee50"}
Jul 24 03:56:49.578949 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:49.578893    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr" podStartSLOduration=1.5788761249999999 podStartE2EDuration="1.578876125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:56:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:56:49.576762956 +0000 UTC m=+2376.337671440" watchObservedRunningTime="2026-07-24 03:56:49.578876125 +0000 UTC m=+2376.339784602"
Jul 24 03:56:52.995838 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:52.995800    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:56:58.555119 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:58.555075    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:58.555119 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:58.555129    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:58.567445 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:58.567416    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:56:58.611985 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:56:58.611959    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:57:07.742959 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.742924    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"]
Jul 24 03:57:07.751619 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.751595    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.755331 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.755302    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b0d347\"/\"openshift-service-ca.crt\""
Jul 24 03:57:07.755767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.755746    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b0d347\"/\"test-model-b0d347-kserve-self-signed-certs\""
Jul 24 03:57:07.756069 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.755822    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b0d347\"/\"default-dockercfg-f8b56\""
Jul 24 03:57:07.756189 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.755883    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b0d347\"/\"kube-root-ca.crt\""
Jul 24 03:57:07.759140 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.758723    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"]
Jul 24 03:57:07.846541 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.846501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-dshm\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.846541 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.846542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-home\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.846804 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.846573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-tmp-dir\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.846804 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.846647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4318cb79-b474-473d-ad60-2df01f071ab0-tls-certs\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.846804 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.846704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-model-cache\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.846804 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.846729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v62ck\" (UniqueName: \"kubernetes.io/projected/4318cb79-b474-473d-ad60-2df01f071ab0-kube-api-access-v62ck\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.947632 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.947586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-tmp-dir\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.947826 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.947638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4318cb79-b474-473d-ad60-2df01f071ab0-tls-certs\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.947826 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.947691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-model-cache\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.947826 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.947721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v62ck\" (UniqueName: \"kubernetes.io/projected/4318cb79-b474-473d-ad60-2df01f071ab0-kube-api-access-v62ck\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.947826 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.947763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-dshm\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.947826 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.947794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-home\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.948108 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.948054    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-tmp-dir\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.948108 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.948068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-model-cache\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.948210 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.948162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-home\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.950148 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.950123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-dshm\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.950347 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.950327    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4318cb79-b474-473d-ad60-2df01f071ab0-tls-certs\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:07.958235 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:07.958215    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v62ck\" (UniqueName: \"kubernetes.io/projected/4318cb79-b474-473d-ad60-2df01f071ab0-kube-api-access-v62ck\") pod \"test-model-b0d347-kserve-69bc5c4c9c-4rrml\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") " pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:08.061488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:08.061392    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:08.195640 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:08.195617    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"]
Jul 24 03:57:08.197330 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:57:08.197302    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4318cb79_b474_473d_ad60_2df01f071ab0.slice/crio-4f070ea9782175ba184716214d66e89ce0002081985edf556868f826b576d50e WatchSource:0}: Error finding container 4f070ea9782175ba184716214d66e89ce0002081985edf556868f826b576d50e: Status 404 returned error can't find the container with id 4f070ea9782175ba184716214d66e89ce0002081985edf556868f826b576d50e
Jul 24 03:57:08.651200 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:08.651159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml" event={"ID":"4318cb79-b474-473d-ad60-2df01f071ab0","Type":"ContainerStarted","Data":"9f3547803ec8786cafe9087fb64b9d27b772422eeb9126a220808c83d7dc1551"}
Jul 24 03:57:08.651200 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:08.651203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml" event={"ID":"4318cb79-b474-473d-ad60-2df01f071ab0","Type":"ContainerStarted","Data":"4f070ea9782175ba184716214d66e89ce0002081985edf556868f826b576d50e"}
Jul 24 03:57:08.676380 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:08.676324    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml" podStartSLOduration=1.6763077690000001 podStartE2EDuration="1.676307769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:57:08.674614154 +0000 UTC m=+2395.435522632" watchObservedRunningTime="2026-07-24 03:57:08.676307769 +0000 UTC m=+2395.437216247"
Jul 24 03:57:13.702760 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:13.702722    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:57:18.062337 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:18.062294    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:18.062337 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:18.062342    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:18.074452 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:18.074422    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:57:18.710845 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:57:18.710817    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:58:24.008707 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:24.008672    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"]
Jul 24 03:58:24.925281 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:24.925239    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"]
Jul 24 03:58:27.799926 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:27.799884    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:58:39.029484 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.029418    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr" podUID="0c321117-b3f8-481e-a03a-cb81daf1986f" containerName="main" containerID="cri-o://7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7" gracePeriod=45
Jul 24 03:58:39.284329 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.284228    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:58:39.354411 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354371    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-dshm\") pod \"0c321117-b3f8-481e-a03a-cb81daf1986f\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") "
Jul 24 03:58:39.354612 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354444    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-model-cache\") pod \"0c321117-b3f8-481e-a03a-cb81daf1986f\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") "
Jul 24 03:58:39.354612 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354482    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wv8r\" (UniqueName: \"kubernetes.io/projected/0c321117-b3f8-481e-a03a-cb81daf1986f-kube-api-access-2wv8r\") pod \"0c321117-b3f8-481e-a03a-cb81daf1986f\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") "
Jul 24 03:58:39.354612 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354499    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-home\") pod \"0c321117-b3f8-481e-a03a-cb81daf1986f\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") "
Jul 24 03:58:39.354612 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354549    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c321117-b3f8-481e-a03a-cb81daf1986f-tls-certs\") pod \"0c321117-b3f8-481e-a03a-cb81daf1986f\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") "
Jul 24 03:58:39.354870 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354615    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-tmp-dir\") pod \"0c321117-b3f8-481e-a03a-cb81daf1986f\" (UID: \"0c321117-b3f8-481e-a03a-cb81daf1986f\") "
Jul 24 03:58:39.354870 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354759    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0c321117-b3f8-481e-a03a-cb81daf1986f" (UID: "0c321117-b3f8-481e-a03a-cb81daf1986f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:58:39.354870 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354776    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-home" (OuterVolumeSpecName: "home") pod "0c321117-b3f8-481e-a03a-cb81daf1986f" (UID: "0c321117-b3f8-481e-a03a-cb81daf1986f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:58:39.355033 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354881    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0c321117-b3f8-481e-a03a-cb81daf1986f" (UID: "0c321117-b3f8-481e-a03a-cb81daf1986f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:58:39.355033 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354905    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:39.355033 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.354925    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:39.356924 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.356891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c321117-b3f8-481e-a03a-cb81daf1986f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0c321117-b3f8-481e-a03a-cb81daf1986f" (UID: "0c321117-b3f8-481e-a03a-cb81daf1986f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:58:39.357045 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.356927    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-dshm" (OuterVolumeSpecName: "dshm") pod "0c321117-b3f8-481e-a03a-cb81daf1986f" (UID: "0c321117-b3f8-481e-a03a-cb81daf1986f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:58:39.357045 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.356935    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c321117-b3f8-481e-a03a-cb81daf1986f-kube-api-access-2wv8r" (OuterVolumeSpecName: "kube-api-access-2wv8r") pod "0c321117-b3f8-481e-a03a-cb81daf1986f" (UID: "0c321117-b3f8-481e-a03a-cb81daf1986f"). InnerVolumeSpecName "kube-api-access-2wv8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:58:39.455609 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.455550    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:39.455609 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.455598    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c321117-b3f8-481e-a03a-cb81daf1986f-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:39.455609 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.455610    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wv8r\" (UniqueName: \"kubernetes.io/projected/0c321117-b3f8-481e-a03a-cb81daf1986f-kube-api-access-2wv8r\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:39.455609 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.455620    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c321117-b3f8-481e-a03a-cb81daf1986f-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:39.944727 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:39.944657    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml" podUID="4318cb79-b474-473d-ad60-2df01f071ab0" containerName="main" containerID="cri-o://9f3547803ec8786cafe9087fb64b9d27b772422eeb9126a220808c83d7dc1551" gracePeriod=45
Jul 24 03:58:40.106883 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.106843    2577 generic.go:356] "Generic (PLEG): container finished" podID="4318cb79-b474-473d-ad60-2df01f071ab0" containerID="9f3547803ec8786cafe9087fb64b9d27b772422eeb9126a220808c83d7dc1551" exitCode=0
Jul 24 03:58:40.107313 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.106913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml" event={"ID":"4318cb79-b474-473d-ad60-2df01f071ab0","Type":"ContainerDied","Data":"9f3547803ec8786cafe9087fb64b9d27b772422eeb9126a220808c83d7dc1551"}
Jul 24 03:58:40.108380 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.108352    2577 generic.go:356] "Generic (PLEG): container finished" podID="0c321117-b3f8-481e-a03a-cb81daf1986f" containerID="7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7" exitCode=0
Jul 24 03:58:40.108513 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.108400    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr" event={"ID":"0c321117-b3f8-481e-a03a-cb81daf1986f","Type":"ContainerDied","Data":"7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7"}
Jul 24 03:58:40.108513 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.108421    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"
Jul 24 03:58:40.108513 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.108437    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr" event={"ID":"0c321117-b3f8-481e-a03a-cb81daf1986f","Type":"ContainerDied","Data":"5d7394e6c59d702774a1b41cc047030688ef9c9ff206c046547dd31b41f8ee50"}
Jul 24 03:58:40.108513 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.108460    2577 scope.go:117] "RemoveContainer" containerID="7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7"
Jul 24 03:58:40.118208 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.118186    2577 scope.go:117] "RemoveContainer" containerID="7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7"
Jul 24 03:58:40.118611 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:58:40.118578    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7\": container with ID starting with 7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7 not found: ID does not exist" containerID="7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7"
Jul 24 03:58:40.118718 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.118621    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7"} err="failed to get container status \"7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7\": rpc error: code = NotFound desc = could not find container \"7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7\": container with ID starting with 7881f41c30d446da74c7db47fe94a95a68165495b15423f6ea45119ec70e6ec7 not found: ID does not exist"
Jul 24 03:58:40.127125 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.127092    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"]
Jul 24 03:58:40.134060 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.134026    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-93dff3/test-model-93dff3-kserve-57445449c9-7wvkr"]
Jul 24 03:58:40.207111 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.207023    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:58:40.363383 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.363342    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v62ck\" (UniqueName: \"kubernetes.io/projected/4318cb79-b474-473d-ad60-2df01f071ab0-kube-api-access-v62ck\") pod \"4318cb79-b474-473d-ad60-2df01f071ab0\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") "
Jul 24 03:58:40.363594 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.363396    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4318cb79-b474-473d-ad60-2df01f071ab0-tls-certs\") pod \"4318cb79-b474-473d-ad60-2df01f071ab0\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") "
Jul 24 03:58:40.363594 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.363458    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-model-cache\") pod \"4318cb79-b474-473d-ad60-2df01f071ab0\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") "
Jul 24 03:58:40.363594 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.363491    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-tmp-dir\") pod \"4318cb79-b474-473d-ad60-2df01f071ab0\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") "
Jul 24 03:58:40.363594 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.363522    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-home\") pod \"4318cb79-b474-473d-ad60-2df01f071ab0\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") "
Jul 24 03:58:40.363594 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.363547    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-dshm\") pod \"4318cb79-b474-473d-ad60-2df01f071ab0\" (UID: \"4318cb79-b474-473d-ad60-2df01f071ab0\") "
Jul 24 03:58:40.363864 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.363814    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-model-cache" (OuterVolumeSpecName: "model-cache") pod "4318cb79-b474-473d-ad60-2df01f071ab0" (UID: "4318cb79-b474-473d-ad60-2df01f071ab0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:58:40.363864 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.363831    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-home" (OuterVolumeSpecName: "home") pod "4318cb79-b474-473d-ad60-2df01f071ab0" (UID: "4318cb79-b474-473d-ad60-2df01f071ab0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:58:40.363864 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.363843    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4318cb79-b474-473d-ad60-2df01f071ab0" (UID: "4318cb79-b474-473d-ad60-2df01f071ab0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:58:40.365920 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.365874    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4318cb79-b474-473d-ad60-2df01f071ab0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4318cb79-b474-473d-ad60-2df01f071ab0" (UID: "4318cb79-b474-473d-ad60-2df01f071ab0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:58:40.365920 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.365886    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-dshm" (OuterVolumeSpecName: "dshm") pod "4318cb79-b474-473d-ad60-2df01f071ab0" (UID: "4318cb79-b474-473d-ad60-2df01f071ab0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:58:40.366076 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.365950    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4318cb79-b474-473d-ad60-2df01f071ab0-kube-api-access-v62ck" (OuterVolumeSpecName: "kube-api-access-v62ck") pod "4318cb79-b474-473d-ad60-2df01f071ab0" (UID: "4318cb79-b474-473d-ad60-2df01f071ab0"). InnerVolumeSpecName "kube-api-access-v62ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:58:40.464796 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.464699    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v62ck\" (UniqueName: \"kubernetes.io/projected/4318cb79-b474-473d-ad60-2df01f071ab0-kube-api-access-v62ck\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:40.464796 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.464733    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4318cb79-b474-473d-ad60-2df01f071ab0-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:40.464796 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.464746    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:40.464796 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.464755    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:40.464796 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.464763    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:40.464796 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.464770    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4318cb79-b474-473d-ad60-2df01f071ab0-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:40.495460 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:40.495419    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:58:41.116764 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:41.116729    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"
Jul 24 03:58:41.116764 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:41.116740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml" event={"ID":"4318cb79-b474-473d-ad60-2df01f071ab0","Type":"ContainerDied","Data":"4f070ea9782175ba184716214d66e89ce0002081985edf556868f826b576d50e"}
Jul 24 03:58:41.117385 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:41.116784    2577 scope.go:117] "RemoveContainer" containerID="9f3547803ec8786cafe9087fb64b9d27b772422eeb9126a220808c83d7dc1551"
Jul 24 03:58:41.140375 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:41.140342    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"]
Jul 24 03:58:41.147855 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:41.147823    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b0d347/test-model-b0d347-kserve-69bc5c4c9c-4rrml"]
Jul 24 03:58:41.643362 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:41.643322    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c321117-b3f8-481e-a03a-cb81daf1986f" path="/var/lib/kubelet/pods/0c321117-b3f8-481e-a03a-cb81daf1986f/volumes"
Jul 24 03:58:41.643738 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:41.643725    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4318cb79-b474-473d-ad60-2df01f071ab0" path="/var/lib/kubelet/pods/4318cb79-b474-473d-ad60-2df01f071ab0/volumes"
Jul 24 03:58:57.433878 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.433829    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"]
Jul 24 03:58:57.434531 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.434099    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47" podUID="bd272f8e-34f1-47c9-9913-ae1819c3fb1c" containerName="maas-api" containerID="cri-o://965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132" gracePeriod=30
Jul 24 03:58:57.681295 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.681248    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:58:57.714062 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.713952    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-485lh\" (UniqueName: \"kubernetes.io/projected/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-kube-api-access-485lh\") pod \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") "
Jul 24 03:58:57.714062 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.714019    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-openshift-service-ca\") pod \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") "
Jul 24 03:58:57.714330 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.714088    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-maas-api-tls\") pod \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\" (UID: \"bd272f8e-34f1-47c9-9913-ae1819c3fb1c\") "
Jul 24 03:58:57.714375 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.714353    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd272f8e-34f1-47c9-9913-ae1819c3fb1c" (UID: "bd272f8e-34f1-47c9-9913-ae1819c3fb1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:58:57.716848 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.716793    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd272f8e-34f1-47c9-9913-ae1819c3fb1c" (UID: "bd272f8e-34f1-47c9-9913-ae1819c3fb1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:58:57.716995 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.716927    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-kube-api-access-485lh" (OuterVolumeSpecName: "kube-api-access-485lh") pod "bd272f8e-34f1-47c9-9913-ae1819c3fb1c" (UID: "bd272f8e-34f1-47c9-9913-ae1819c3fb1c"). InnerVolumeSpecName "kube-api-access-485lh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:58:57.815168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.815136    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-485lh\" (UniqueName: \"kubernetes.io/projected/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-kube-api-access-485lh\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:57.815168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.815163    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:57.815168 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:57.815174    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd272f8e-34f1-47c9-9913-ae1819c3fb1c-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:58:58.203623 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:58.203585    2577 generic.go:356] "Generic (PLEG): container finished" podID="bd272f8e-34f1-47c9-9913-ae1819c3fb1c" containerID="965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132" exitCode=0
Jul 24 03:58:58.203836 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:58.203644    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"
Jul 24 03:58:58.203836 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:58.203681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47" event={"ID":"bd272f8e-34f1-47c9-9913-ae1819c3fb1c","Type":"ContainerDied","Data":"965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132"}
Jul 24 03:58:58.203836 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:58.203780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47" event={"ID":"bd272f8e-34f1-47c9-9913-ae1819c3fb1c","Type":"ContainerDied","Data":"919e57c992522e1e9b1d09e33bf0eff99aaa6ca4c0bab1506635243cd966a04a"}
Jul 24 03:58:58.203836 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:58.203809    2577 scope.go:117] "RemoveContainer" containerID="965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132"
Jul 24 03:58:58.213293 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:58.213244    2577 scope.go:117] "RemoveContainer" containerID="965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132"
Jul 24 03:58:58.213594 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:58:58.213573    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132\": container with ID starting with 965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132 not found: ID does not exist" containerID="965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132"
Jul 24 03:58:58.213646 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:58.213605    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132"} err="failed to get container status \"965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132\": rpc error: code = NotFound desc = could not find container \"965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132\": container with ID starting with 965363cf3bf5c01dac0730460b59a1a7b545b340d97e84e2853023635ab3d132 not found: ID does not exist"
Jul 24 03:58:58.225342 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:58.225309    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"]
Jul 24 03:58:58.232007 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:58.231975    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b0d347-789d8fc655-fwk47"]
Jul 24 03:58:59.643018 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:58:59.642979    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd272f8e-34f1-47c9-9913-ae1819c3fb1c" path="/var/lib/kubelet/pods/bd272f8e-34f1-47c9-9913-ae1819c3fb1c/volumes"
Jul 24 03:59:00.203402 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:00.203364    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:59:13.349702 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.349611    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"]
Jul 24 03:59:13.350151 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.350075    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c321117-b3f8-481e-a03a-cb81daf1986f" containerName="main"
Jul 24 03:59:13.350151 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.350094    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c321117-b3f8-481e-a03a-cb81daf1986f" containerName="main"
Jul 24 03:59:13.350151 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.350107    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4318cb79-b474-473d-ad60-2df01f071ab0" containerName="main"
Jul 24 03:59:13.350151 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.350115    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4318cb79-b474-473d-ad60-2df01f071ab0" containerName="main"
Jul 24 03:59:13.350151 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.350141    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd272f8e-34f1-47c9-9913-ae1819c3fb1c" containerName="maas-api"
Jul 24 03:59:13.350151 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.350150    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd272f8e-34f1-47c9-9913-ae1819c3fb1c" containerName="maas-api"
Jul 24 03:59:13.350488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.350240    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c321117-b3f8-481e-a03a-cb81daf1986f" containerName="main"
Jul 24 03:59:13.350488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.350254    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd272f8e-34f1-47c9-9913-ae1819c3fb1c" containerName="maas-api"
Jul 24 03:59:13.350488 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.350288    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4318cb79-b474-473d-ad60-2df01f071ab0" containerName="main"
Jul 24 03:59:13.354810 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.354787    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.357366 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.357346    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-58nst\""
Jul 24 03:59:13.357491 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.357347    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f822cd-serving-cert\""
Jul 24 03:59:13.358192 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.358176    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 03:59:13.358285 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.358207    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:59:13.365767 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.365739    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"]
Jul 24 03:59:13.443663 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.443627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2hrg\" (UniqueName: \"kubernetes.io/projected/9f8820e4-baba-4583-b1b3-645119ec60df-kube-api-access-c2hrg\") pod \"maas-api-e2e-api-a-f822cd-d6789c9c4-76dck\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.443663 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.443667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8820e4-baba-4583-b1b3-645119ec60df-openshift-service-ca\") pod \"maas-api-e2e-api-a-f822cd-d6789c9c4-76dck\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.443900 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.443694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f8820e4-baba-4583-b1b3-645119ec60df-maas-api-tls\") pod \"maas-api-e2e-api-a-f822cd-d6789c9c4-76dck\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.544624 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.544587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2hrg\" (UniqueName: \"kubernetes.io/projected/9f8820e4-baba-4583-b1b3-645119ec60df-kube-api-access-c2hrg\") pod \"maas-api-e2e-api-a-f822cd-d6789c9c4-76dck\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.544624 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.544624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8820e4-baba-4583-b1b3-645119ec60df-openshift-service-ca\") pod \"maas-api-e2e-api-a-f822cd-d6789c9c4-76dck\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.544920 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.544649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f8820e4-baba-4583-b1b3-645119ec60df-maas-api-tls\") pod \"maas-api-e2e-api-a-f822cd-d6789c9c4-76dck\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.549078 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.549044    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:59:13.549205 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.549131    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f822cd-serving-cert\""
Jul 24 03:59:13.555568 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.555542    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8820e4-baba-4583-b1b3-645119ec60df-openshift-service-ca\") pod \"maas-api-e2e-api-a-f822cd-d6789c9c4-76dck\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.555852 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.555648    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 03:59:13.557579 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.557558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f8820e4-baba-4583-b1b3-645119ec60df-maas-api-tls\") pod \"maas-api-e2e-api-a-f822cd-d6789c9c4-76dck\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.565542 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.565519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2hrg\" (UniqueName: \"kubernetes.io/projected/9f8820e4-baba-4583-b1b3-645119ec60df-kube-api-access-c2hrg\") pod \"maas-api-e2e-api-a-f822cd-d6789c9c4-76dck\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.668093 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.668065    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-58nst\""
Jul 24 03:59:13.674927 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.674899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:13.816198 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:13.816147    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"]
Jul 24 03:59:13.818561 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:59:13.818532    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8820e4_baba_4583_b1b3_645119ec60df.slice/crio-95a4911f57fb630e95f6b2cf9d617a9b53409ef911b755be53ebfd8ebe420b54 WatchSource:0}: Error finding container 95a4911f57fb630e95f6b2cf9d617a9b53409ef911b755be53ebfd8ebe420b54: Status 404 returned error can't find the container with id 95a4911f57fb630e95f6b2cf9d617a9b53409ef911b755be53ebfd8ebe420b54
Jul 24 03:59:14.285199 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:14.285164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck" event={"ID":"9f8820e4-baba-4583-b1b3-645119ec60df","Type":"ContainerStarted","Data":"95a4911f57fb630e95f6b2cf9d617a9b53409ef911b755be53ebfd8ebe420b54"}
Jul 24 03:59:15.291636 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:15.291601    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck" event={"ID":"9f8820e4-baba-4583-b1b3-645119ec60df","Type":"ContainerStarted","Data":"2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d"}
Jul 24 03:59:15.292040 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:15.291766    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:15.313403 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:15.313347    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck" podStartSLOduration=1.816354534 podStartE2EDuration="2.313328485s" totalImagesPullingTime="496.973951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:59:15.312323964 +0000 UTC m=+2522.073232449" watchObservedRunningTime="2026-07-24 03:59:15.313328485 +0000 UTC m=+2522.074236964"
Jul 24 03:59:20.792848 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:20.792794    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:59:21.302604 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.302573    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:21.327117 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.327082    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"]
Jul 24 03:59:21.331052 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.331026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.334260 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.334238    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b23325-serving-cert\""
Jul 24 03:59:21.347808 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.347769    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"]
Jul 24 03:59:21.411663 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.411619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-maas-api-tls\") pod \"maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.411663 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.411666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt7dp\" (UniqueName: \"kubernetes.io/projected/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-kube-api-access-zt7dp\") pod \"maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.411901 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.411796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-openshift-service-ca\") pod \"maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.512381 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.512344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-maas-api-tls\") pod \"maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.512381 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.512386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt7dp\" (UniqueName: \"kubernetes.io/projected/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-kube-api-access-zt7dp\") pod \"maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.512649 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.512447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-openshift-service-ca\") pod \"maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.513236 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.513213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-openshift-service-ca\") pod \"maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.515360 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.515333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-maas-api-tls\") pod \"maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.523613 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.523586    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt7dp\" (UniqueName: \"kubernetes.io/projected/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-kube-api-access-zt7dp\") pod \"maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.642505 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.642430    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:21.784031 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:21.784004    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"]
Jul 24 03:59:21.786183 ip-10-0-132-49 kubenswrapper[2577]: W0724 03:59:21.786154    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc019da6_b6a9_4c9a_b1fe_e96fa3539dbc.slice/crio-6e67d5e2f86534f19af40ed5b89e14e5c6402b17c00d28ee1228847004699292 WatchSource:0}: Error finding container 6e67d5e2f86534f19af40ed5b89e14e5c6402b17c00d28ee1228847004699292: Status 404 returned error can't find the container with id 6e67d5e2f86534f19af40ed5b89e14e5c6402b17c00d28ee1228847004699292
Jul 24 03:59:22.330599 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:22.330561    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq" event={"ID":"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc","Type":"ContainerStarted","Data":"0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229"}
Jul 24 03:59:22.331028 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:22.330624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq" event={"ID":"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc","Type":"ContainerStarted","Data":"6e67d5e2f86534f19af40ed5b89e14e5c6402b17c00d28ee1228847004699292"}
Jul 24 03:59:22.331028 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:22.330703    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:22.349212 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:22.349153    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq" podStartSLOduration=1.0351203390000001 podStartE2EDuration="1.349136011s" totalImagesPullingTime="314.015672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:59:22.347761596 +0000 UTC m=+2529.108670074" watchObservedRunningTime="2026-07-24 03:59:22.349136011 +0000 UTC m=+2529.110044490"
Jul 24 03:59:28.340240 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:28.340204    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 03:59:35.900765 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:35.900726    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 03:59:49.540161 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:49.540110    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"]
Jul 24 03:59:49.540714 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:49.540460    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck" podUID="9f8820e4-baba-4583-b1b3-645119ec60df" containerName="maas-api" containerID="cri-o://2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d" gracePeriod=30
Jul 24 03:59:49.782982 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:49.782954    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:49.962821 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:49.962786    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f8820e4-baba-4583-b1b3-645119ec60df-maas-api-tls\") pod \"9f8820e4-baba-4583-b1b3-645119ec60df\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") "
Jul 24 03:59:49.963030 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:49.962886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8820e4-baba-4583-b1b3-645119ec60df-openshift-service-ca\") pod \"9f8820e4-baba-4583-b1b3-645119ec60df\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") "
Jul 24 03:59:49.963030 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:49.962930    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2hrg\" (UniqueName: \"kubernetes.io/projected/9f8820e4-baba-4583-b1b3-645119ec60df-kube-api-access-c2hrg\") pod \"9f8820e4-baba-4583-b1b3-645119ec60df\" (UID: \"9f8820e4-baba-4583-b1b3-645119ec60df\") "
Jul 24 03:59:49.963346 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:49.963315    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f8820e4-baba-4583-b1b3-645119ec60df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f8820e4-baba-4583-b1b3-645119ec60df" (UID: "9f8820e4-baba-4583-b1b3-645119ec60df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:59:49.965124 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:49.965097    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f8820e4-baba-4583-b1b3-645119ec60df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f8820e4-baba-4583-b1b3-645119ec60df" (UID: "9f8820e4-baba-4583-b1b3-645119ec60df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:59:49.965361 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:49.965343    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8820e4-baba-4583-b1b3-645119ec60df-kube-api-access-c2hrg" (OuterVolumeSpecName: "kube-api-access-c2hrg") pod "9f8820e4-baba-4583-b1b3-645119ec60df" (UID: "9f8820e4-baba-4583-b1b3-645119ec60df"). InnerVolumeSpecName "kube-api-access-c2hrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:59:50.063908 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.063858    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2hrg\" (UniqueName: \"kubernetes.io/projected/9f8820e4-baba-4583-b1b3-645119ec60df-kube-api-access-c2hrg\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:59:50.063908 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.063903    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f8820e4-baba-4583-b1b3-645119ec60df-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:59:50.063908 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.063913    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f8820e4-baba-4583-b1b3-645119ec60df-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 03:59:50.482467 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.482425    2577 generic.go:356] "Generic (PLEG): container finished" podID="9f8820e4-baba-4583-b1b3-645119ec60df" containerID="2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d" exitCode=0
Jul 24 03:59:50.482686 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.482476    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"
Jul 24 03:59:50.482686 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.482520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck" event={"ID":"9f8820e4-baba-4583-b1b3-645119ec60df","Type":"ContainerDied","Data":"2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d"}
Jul 24 03:59:50.482686 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.482589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck" event={"ID":"9f8820e4-baba-4583-b1b3-645119ec60df","Type":"ContainerDied","Data":"95a4911f57fb630e95f6b2cf9d617a9b53409ef911b755be53ebfd8ebe420b54"}
Jul 24 03:59:50.482686 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.482621    2577 scope.go:117] "RemoveContainer" containerID="2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d"
Jul 24 03:59:50.492525 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.492502    2577 scope.go:117] "RemoveContainer" containerID="2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d"
Jul 24 03:59:50.492827 ip-10-0-132-49 kubenswrapper[2577]: E0724 03:59:50.492800    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d\": container with ID starting with 2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d not found: ID does not exist" containerID="2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d"
Jul 24 03:59:50.492899 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.492836    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d"} err="failed to get container status \"2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d\": rpc error: code = NotFound desc = could not find container \"2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d\": container with ID starting with 2c9095bc666d9660af8bd754495a8a8465f68db42c5652eda55839e03a7bd06d not found: ID does not exist"
Jul 24 03:59:50.504675 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.504642    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"]
Jul 24 03:59:50.508811 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:50.508781    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f822cd-d6789c9c4-76dck"]
Jul 24 03:59:51.642525 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:51.642492    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8820e4-baba-4583-b1b3-645119ec60df" path="/var/lib/kubelet/pods/9f8820e4-baba-4583-b1b3-645119ec60df/volumes"
Jul 24 03:59:52.806799 ip-10-0-132-49 kubenswrapper[2577]: I0724 03:59:52.806761    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:00:00.976431 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:00.976397    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"]
Jul 24 04:00:00.976935 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:00.976715    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8820e4-baba-4583-b1b3-645119ec60df" containerName="maas-api"
Jul 24 04:00:00.976935 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:00.976725    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8820e4-baba-4583-b1b3-645119ec60df" containerName="maas-api"
Jul 24 04:00:00.976935 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:00.976784    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8820e4-baba-4583-b1b3-645119ec60df" containerName="maas-api"
Jul 24 04:00:00.981013 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:00.980993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"
Jul 24 04:00:00.984837 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:00.984802    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 04:00:00.992031 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:00.992004    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"]
Jul 24 04:00:01.053308 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:01.053232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12ef1e4-d69d-4fd9-b485-54c271715b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b23325-gxwc6\" (UID: \"e12ef1e4-d69d-4fd9-b485-54c271715b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"
Jul 24 04:00:01.154491 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:01.154437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12ef1e4-d69d-4fd9-b485-54c271715b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b23325-gxwc6\" (UID: \"e12ef1e4-d69d-4fd9-b485-54c271715b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"
Jul 24 04:00:01.155067 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:01.155046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12ef1e4-d69d-4fd9-b485-54c271715b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b23325-gxwc6\" (UID: \"e12ef1e4-d69d-4fd9-b485-54c271715b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"
Jul 24 04:00:01.293016 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:01.292925    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"
Jul 24 04:00:01.434438 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:01.434409    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"]
Jul 24 04:00:01.436221 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:00:01.436188    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode12ef1e4_d69d_4fd9_b485_54c271715b63.slice/crio-8a31e1489982ded4d9c2bb1e3538b85859fd3480520703e8feacd0fad9b378d8 WatchSource:0}: Error finding container 8a31e1489982ded4d9c2bb1e3538b85859fd3480520703e8feacd0fad9b378d8: Status 404 returned error can't find the container with id 8a31e1489982ded4d9c2bb1e3538b85859fd3480520703e8feacd0fad9b378d8
Jul 24 04:00:01.538120 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:01.538082    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6" event={"ID":"e12ef1e4-d69d-4fd9-b485-54c271715b63","Type":"ContainerStarted","Data":"8a31e1489982ded4d9c2bb1e3538b85859fd3480520703e8feacd0fad9b378d8"}
Jul 24 04:00:18.625194 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:18.625102    2577 generic.go:356] "Generic (PLEG): container finished" podID="e12ef1e4-d69d-4fd9-b485-54c271715b63" containerID="c7d68eabdc0bd9d0ea7728c9cb1dc84c1c91d8d93f29576a99bd10617a6fb60c" exitCode=0
Jul 24 04:00:18.625194 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:18.625180    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6" event={"ID":"e12ef1e4-d69d-4fd9-b485-54c271715b63","Type":"ContainerDied","Data":"c7d68eabdc0bd9d0ea7728c9cb1dc84c1c91d8d93f29576a99bd10617a6fb60c"}
Jul 24 04:00:19.764585 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:19.764554    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"
Jul 24 04:00:19.815995 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:19.815958    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12ef1e4-d69d-4fd9-b485-54c271715b63-maas-api-service-ca\") pod \"e12ef1e4-d69d-4fd9-b485-54c271715b63\" (UID: \"e12ef1e4-d69d-4fd9-b485-54c271715b63\") "
Jul 24 04:00:19.816355 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:19.816332    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e12ef1e4-d69d-4fd9-b485-54c271715b63-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e12ef1e4-d69d-4fd9-b485-54c271715b63" (UID: "e12ef1e4-d69d-4fd9-b485-54c271715b63"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:00:19.917121 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:19.917080    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12ef1e4-d69d-4fd9-b485-54c271715b63-maas-api-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:00:20.636420 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:20.636380    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"
Jul 24 04:00:20.636595 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:20.636382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6" event={"ID":"e12ef1e4-d69d-4fd9-b485-54c271715b63","Type":"ContainerDied","Data":"8a31e1489982ded4d9c2bb1e3538b85859fd3480520703e8feacd0fad9b378d8"}
Jul 24 04:00:20.636595 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:20.636515    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a31e1489982ded4d9c2bb1e3538b85859fd3480520703e8feacd0fad9b378d8"
Jul 24 04:00:26.799438 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:26.799399    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"]
Jul 24 04:00:26.799943 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:26.799700    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq" podUID="fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc" containerName="maas-api" containerID="cri-o://0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229" gracePeriod=30
Jul 24 04:00:27.044480 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.044449    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 04:00:27.178504 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.178471    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-maas-api-tls\") pod \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") "
Jul 24 04:00:27.178693 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.178534    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-openshift-service-ca\") pod \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") "
Jul 24 04:00:27.178693 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.178572    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt7dp\" (UniqueName: \"kubernetes.io/projected/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-kube-api-access-zt7dp\") pod \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\" (UID: \"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc\") "
Jul 24 04:00:27.178941 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.178920    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc" (UID: "fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:00:27.180810 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.180786    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc" (UID: "fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:00:27.180810 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.180782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-kube-api-access-zt7dp" (OuterVolumeSpecName: "kube-api-access-zt7dp") pod "fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc" (UID: "fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc"). InnerVolumeSpecName "kube-api-access-zt7dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:00:27.279721 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.279676    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:00:27.279721 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.279711    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:00:27.279721 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.279722    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt7dp\" (UniqueName: \"kubernetes.io/projected/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc-kube-api-access-zt7dp\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:00:27.673291 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.673231    2577 generic.go:356] "Generic (PLEG): container finished" podID="fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc" containerID="0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229" exitCode=0
Jul 24 04:00:27.673459 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.673302    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"
Jul 24 04:00:27.673459 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.673298    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq" event={"ID":"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc","Type":"ContainerDied","Data":"0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229"}
Jul 24 04:00:27.673459 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.673420    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq" event={"ID":"fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc","Type":"ContainerDied","Data":"6e67d5e2f86534f19af40ed5b89e14e5c6402b17c00d28ee1228847004699292"}
Jul 24 04:00:27.673459 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.673453    2577 scope.go:117] "RemoveContainer" containerID="0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229"
Jul 24 04:00:27.684076 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.684052    2577 scope.go:117] "RemoveContainer" containerID="0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229"
Jul 24 04:00:27.684381 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:00:27.684358    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229\": container with ID starting with 0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229 not found: ID does not exist" containerID="0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229"
Jul 24 04:00:27.684444 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.684395    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229"} err="failed to get container status \"0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229\": rpc error: code = NotFound desc = could not find container \"0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229\": container with ID starting with 0712272de0abc5f7d104d874c90c448874cf259e1f3d7c5d2479e71cb7a9e229 not found: ID does not exist"
Jul 24 04:00:27.693454 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.693427    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"]
Jul 24 04:00:27.695189 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:27.695166    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b23325-96d6cbb47-h5mfq"]
Jul 24 04:00:29.643450 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:29.643412    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc" path="/var/lib/kubelet/pods/fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc/volumes"
Jul 24 04:00:31.071710 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:31.071667    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"]
Jul 24 04:00:31.074747 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:31.074712    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b23325-gxwc6"]
Jul 24 04:00:31.642732 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:31.642699    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e12ef1e4-d69d-4fd9-b485-54c271715b63" path="/var/lib/kubelet/pods/e12ef1e4-d69d-4fd9-b485-54c271715b63/volumes"
Jul 24 04:00:31.797982 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:31.797944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:00:41.799840 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.799808    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"]
Jul 24 04:00:41.800236 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.800146    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc" containerName="maas-api"
Jul 24 04:00:41.800236 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.800159    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc" containerName="maas-api"
Jul 24 04:00:41.800236 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.800174    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e12ef1e4-d69d-4fd9-b485-54c271715b63" containerName="revoke-keys"
Jul 24 04:00:41.800236 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.800182    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e12ef1e4-d69d-4fd9-b485-54c271715b63" containerName="revoke-keys"
Jul 24 04:00:41.800236 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.800235    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc019da6-b6a9-4c9a-b1fe-e96fa3539dbc" containerName="maas-api"
Jul 24 04:00:41.800479 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.800245    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e12ef1e4-d69d-4fd9-b485-54c271715b63" containerName="revoke-keys"
Jul 24 04:00:41.804552 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.804534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:41.808405 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.808382    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 04:00:41.808529 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.808417    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 04:00:41.809587 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.809570    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-58nst\""
Jul 24 04:00:41.809658 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.809590    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-5b56b9-serving-cert\""
Jul 24 04:00:41.819644 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.819617    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"]
Jul 24 04:00:41.914177 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.914143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdfcdf50-4c24-44cc-b550-7e10acede653-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:41.914370 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.914277    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdfcdf50-4c24-44cc-b550-7e10acede653-maas-api-tls\") pod \"maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:41.914370 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:41.914331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6fvn\" (UniqueName: \"kubernetes.io/projected/cdfcdf50-4c24-44cc-b550-7e10acede653-kube-api-access-x6fvn\") pod \"maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:42.015143 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:42.015100    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdfcdf50-4c24-44cc-b550-7e10acede653-maas-api-tls\") pod \"maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:42.015143 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:42.015144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6fvn\" (UniqueName: \"kubernetes.io/projected/cdfcdf50-4c24-44cc-b550-7e10acede653-kube-api-access-x6fvn\") pod \"maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:42.015473 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:42.015186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdfcdf50-4c24-44cc-b550-7e10acede653-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:42.016112 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:42.016086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdfcdf50-4c24-44cc-b550-7e10acede653-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:42.017969 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:42.017943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdfcdf50-4c24-44cc-b550-7e10acede653-maas-api-tls\") pod \"maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:42.025655 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:42.025627    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6fvn\" (UniqueName: \"kubernetes.io/projected/cdfcdf50-4c24-44cc-b550-7e10acede653-kube-api-access-x6fvn\") pod \"maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:42.114634 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:42.114537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:42.255939 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:42.255910    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"]
Jul 24 04:00:42.257779 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:00:42.257748    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdfcdf50_4c24_44cc_b550_7e10acede653.slice/crio-70e3b204cabdf48456f20e4bbbec36fa87dac8d463c787cc40b3a44dd5a136ec WatchSource:0}: Error finding container 70e3b204cabdf48456f20e4bbbec36fa87dac8d463c787cc40b3a44dd5a136ec: Status 404 returned error can't find the container with id 70e3b204cabdf48456f20e4bbbec36fa87dac8d463c787cc40b3a44dd5a136ec
Jul 24 04:00:42.749698 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:42.749659    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf" event={"ID":"cdfcdf50-4c24-44cc-b550-7e10acede653","Type":"ContainerStarted","Data":"70e3b204cabdf48456f20e4bbbec36fa87dac8d463c787cc40b3a44dd5a136ec"}
Jul 24 04:00:43.762432 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:43.762397    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf" event={"ID":"cdfcdf50-4c24-44cc-b550-7e10acede653","Type":"ContainerStarted","Data":"da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2"}
Jul 24 04:00:43.762852 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:43.762520    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:43.789927 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:43.789874    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf" podStartSLOduration=2.067746093 podStartE2EDuration="2.789857931s" totalImagesPullingTime="722.111838ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:00:43.787810103 +0000 UTC m=+2610.548718584" watchObservedRunningTime="2026-07-24 04:00:43.789857931 +0000 UTC m=+2610.550766389"
Jul 24 04:00:46.494054 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:46.494013    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:00:49.772613 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:49.772585    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:00:49.861726 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:49.861689    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"]
Jul 24 04:00:49.870888 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:49.870858    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:49.873495 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:49.873463    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-f701e0-serving-cert\""
Jul 24 04:00:49.874792 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:49.874762    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"]
Jul 24 04:00:49.983999 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:49.983961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:49.984180 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:49.984018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-maas-api-tls\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:49.984180 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:49.984069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk8bp\" (UniqueName: \"kubernetes.io/projected/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-kube-api-access-gk8bp\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:50.084599 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:50.084507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-maas-api-tls\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:50.084599 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:50.084549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk8bp\" (UniqueName: \"kubernetes.io/projected/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-kube-api-access-gk8bp\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:50.084791 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:50.084615    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:50.084791 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:00:50.084657    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-serving-cert: secret "maas-api-e2e-shared-b-f701e0-serving-cert" not found
Jul 24 04:00:50.084791 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:00:50.084720    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-maas-api-tls podName:eaa5f0d2-34c9-4c89-be95-b3c81768fe2b nodeName:}" failed. No retries permitted until 2026-07-24 04:00:50.58470236 +0000 UTC m=+2617.345610820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-maas-api-tls") pod "maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn" (UID: "eaa5f0d2-34c9-4c89-be95-b3c81768fe2b") : secret "maas-api-e2e-shared-b-f701e0-serving-cert" not found
Jul 24 04:00:50.085307 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:50.085289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:50.118078 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:50.118038    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk8bp\" (UniqueName: \"kubernetes.io/projected/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-kube-api-access-gk8bp\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:50.590098 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:50.590055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-maas-api-tls\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:50.592948 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:50.592918    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-maas-api-tls\") pod \"maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:50.783031 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:50.782992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:50.921912 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:50.921881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"]
Jul 24 04:00:50.924024 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:00:50.923976    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa5f0d2_34c9_4c89_be95_b3c81768fe2b.slice/crio-4d05f8f94ebf6c47a1583ab22f0d3a8708faf1928686800ba301469258dae1a4 WatchSource:0}: Error finding container 4d05f8f94ebf6c47a1583ab22f0d3a8708faf1928686800ba301469258dae1a4: Status 404 returned error can't find the container with id 4d05f8f94ebf6c47a1583ab22f0d3a8708faf1928686800ba301469258dae1a4
Jul 24 04:00:51.802335 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:51.802292    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn" event={"ID":"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b","Type":"ContainerStarted","Data":"4d05f8f94ebf6c47a1583ab22f0d3a8708faf1928686800ba301469258dae1a4"}
Jul 24 04:00:52.809901 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:52.809812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn" event={"ID":"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b","Type":"ContainerStarted","Data":"87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947"}
Jul 24 04:00:52.809901 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:52.809876    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:00:52.827927 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:52.827865    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn" podStartSLOduration=2.196996824 podStartE2EDuration="3.827841502s" totalImagesPullingTime="1.630844678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:00:52.827674465 +0000 UTC m=+2619.588582961" watchObservedRunningTime="2026-07-24 04:00:52.827841502 +0000 UTC m=+2619.588749985"
Jul 24 04:00:58.313018 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:58.312983    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:00:58.822406 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:00:58.822373    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:01:15.236668 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.236634    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"]
Jul 24 04:01:15.241447 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.241428    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.247493 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.247462    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5b56b9\"/\"openshift-service-ca.crt\""
Jul 24 04:01:15.247493 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.247461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5b56b9\"/\"default-dockercfg-6gjsc\""
Jul 24 04:01:15.247712 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.247462    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5b56b9\"/\"auth-test-model-5b56b9-kserve-self-signed-certs\""
Jul 24 04:01:15.247712 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.247529    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5b56b9\"/\"kube-root-ca.crt\""
Jul 24 04:01:15.256784 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.256753    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"]
Jul 24 04:01:15.280887 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.280853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tls-certs\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.280887 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.280896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4p2c\" (UniqueName: \"kubernetes.io/projected/21db5ab0-1de4-44b2-a926-6c18a6f2993a-kube-api-access-r4p2c\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.281158 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.280965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-home\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.281158 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.280997    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-dshm\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.281158 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.281020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-model-cache\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.281158 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.281048    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tmp-dir\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.382450 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.382399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-home\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.382450 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.382454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-dshm\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.382720 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.382480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-model-cache\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.382720 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.382510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tmp-dir\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.382720 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.382608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tls-certs\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.382720 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.382630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4p2c\" (UniqueName: \"kubernetes.io/projected/21db5ab0-1de4-44b2-a926-6c18a6f2993a-kube-api-access-r4p2c\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.382921 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.382867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-home\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.382921 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.382905    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-model-cache\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.383034 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.382949    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tmp-dir\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.385005 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.384975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-dshm\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.385169 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.385152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tls-certs\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.391151 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.391127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4p2c\" (UniqueName: \"kubernetes.io/projected/21db5ab0-1de4-44b2-a926-6c18a6f2993a-kube-api-access-r4p2c\") pod \"auth-test-model-5b56b9-kserve-759f67b77f-kc9nl\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") " pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.551445 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.551339    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:15.692002 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.691971    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"]
Jul 24 04:01:15.694290 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:01:15.694239    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21db5ab0_1de4_44b2_a926_6c18a6f2993a.slice/crio-8f1f1ac33e43b481553fcfb96772d7cef2eec96a2ea0ad9b7a43d1e5829b4f29 WatchSource:0}: Error finding container 8f1f1ac33e43b481553fcfb96772d7cef2eec96a2ea0ad9b7a43d1e5829b4f29: Status 404 returned error can't find the container with id 8f1f1ac33e43b481553fcfb96772d7cef2eec96a2ea0ad9b7a43d1e5829b4f29
Jul 24 04:01:15.929596 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.929558    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl" event={"ID":"21db5ab0-1de4-44b2-a926-6c18a6f2993a","Type":"ContainerStarted","Data":"2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16"}
Jul 24 04:01:15.929751 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.929608    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl" event={"ID":"21db5ab0-1de4-44b2-a926-6c18a6f2993a","Type":"ContainerStarted","Data":"8f1f1ac33e43b481553fcfb96772d7cef2eec96a2ea0ad9b7a43d1e5829b4f29"}
Jul 24 04:01:15.950499 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:15.950443    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl" podStartSLOduration=0.950423887 podStartE2EDuration="950.423887ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:01:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:01:15.948498642 +0000 UTC m=+2642.709407124" watchObservedRunningTime="2026-07-24 04:01:15.950423887 +0000 UTC m=+2642.711332367"
Jul 24 04:01:18.706000 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:18.705960    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:01:25.552028 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:25.551986    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:25.552028 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:25.552037    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:25.565120 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:25.565089    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:25.994398 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:25.994360    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:01:45.034434 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.034400    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"]
Jul 24 04:01:45.037849 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.037825    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.040491 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.040466    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"openshift-service-ca.crt\""
Jul 24 04:01:45.040491 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.040485    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"kube-root-ca.crt\""
Jul 24 04:01:45.040680 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.040505    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"default-dockercfg-ttgsc\""
Jul 24 04:01:45.041503 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.041484    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"auth-test-model-f701e0-kserve-self-signed-certs\""
Jul 24 04:01:45.048880 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.048854    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"]
Jul 24 04:01:45.163717 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.163674    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-tmp-dir\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.163931 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.163735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-model-cache\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.163931 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.163805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-dshm\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.163931 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.163839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-home\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.163931 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.163876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5538ba50-2405-485b-805e-1ec8a2c2a019-tls-certs\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.164108 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.163933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7ggv\" (UniqueName: \"kubernetes.io/projected/5538ba50-2405-485b-805e-1ec8a2c2a019-kube-api-access-h7ggv\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.265424 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.265375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7ggv\" (UniqueName: \"kubernetes.io/projected/5538ba50-2405-485b-805e-1ec8a2c2a019-kube-api-access-h7ggv\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.265424 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.265425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-tmp-dir\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.265694 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.265466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-model-cache\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.265694 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.265500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-dshm\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.265694 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.265528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-home\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.265694 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.265573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5538ba50-2405-485b-805e-1ec8a2c2a019-tls-certs\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.266319 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.266255    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-tmp-dir\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.266521 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.266345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-model-cache\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.266593 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.266558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-home\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.269259 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.269222    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5538ba50-2405-485b-805e-1ec8a2c2a019-tls-certs\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.274942 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.274895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-dshm\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.276464 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.276441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7ggv\" (UniqueName: \"kubernetes.io/projected/5538ba50-2405-485b-805e-1ec8a2c2a019-kube-api-access-h7ggv\") pod \"auth-test-model-f701e0-kserve-55977d7f-p9k6t\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") " pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.348255 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.348161    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:45.491077 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:45.491037    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"]
Jul 24 04:01:45.495034 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:01:45.495003    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5538ba50_2405_485b_805e_1ec8a2c2a019.slice/crio-ab31681eb196166fe4388a1b4416b480e7cd2fcf432b84a0fd42d1830f8853eb WatchSource:0}: Error finding container ab31681eb196166fe4388a1b4416b480e7cd2fcf432b84a0fd42d1830f8853eb: Status 404 returned error can't find the container with id ab31681eb196166fe4388a1b4416b480e7cd2fcf432b84a0fd42d1830f8853eb
Jul 24 04:01:46.083160 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:46.083125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t" event={"ID":"5538ba50-2405-485b-805e-1ec8a2c2a019","Type":"ContainerStarted","Data":"558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213"}
Jul 24 04:01:46.083160 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:46.083168    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t" event={"ID":"5538ba50-2405-485b-805e-1ec8a2c2a019","Type":"ContainerStarted","Data":"ab31681eb196166fe4388a1b4416b480e7cd2fcf432b84a0fd42d1830f8853eb"}
Jul 24 04:01:46.102158 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:46.102097    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t" podStartSLOduration=1.102079772 podStartE2EDuration="1.102079772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:01:46.09976495 +0000 UTC m=+2672.860673430" watchObservedRunningTime="2026-07-24 04:01:46.102079772 +0000 UTC m=+2672.862988251"
Jul 24 04:01:48.492136 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:48.492093    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:01:55.348952 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:55.348897    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:55.348952 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:55.348955    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:55.361412 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:55.361380    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:01:56.144572 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:01:56.144542    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:02:03.382204 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:02:03.382164    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:02:18.686217 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:02:18.686133    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:02:38.185788 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:02:38.185741    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:02:55.889877 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:02:55.889840    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:03:13.287628 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:03:13.287588    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:03:30.001968 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:03:30.001927    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:03:43.592426 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:03:43.592332    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:03:48.275360 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:03:48.275315    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"]
Jul 24 04:03:49.137321 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:03:49.137289    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"]
Jul 24 04:04:01.385664 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:01.385624    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:04:03.295601 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.295550    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl" podUID="21db5ab0-1de4-44b2-a926-6c18a6f2993a" containerName="main" containerID="cri-o://2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16" gracePeriod=45
Jul 24 04:04:03.551543 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.551475    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:04:03.633254 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633217    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-model-cache\") pod \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") "
Jul 24 04:04:03.633254 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633274    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-home\") pod \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") "
Jul 24 04:04:03.633544 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633308    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-dshm\") pod \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") "
Jul 24 04:04:03.633544 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633328    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tls-certs\") pod \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") "
Jul 24 04:04:03.633544 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633344    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tmp-dir\") pod \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") "
Jul 24 04:04:03.633544 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633364    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4p2c\" (UniqueName: \"kubernetes.io/projected/21db5ab0-1de4-44b2-a926-6c18a6f2993a-kube-api-access-r4p2c\") pod \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\" (UID: \"21db5ab0-1de4-44b2-a926-6c18a6f2993a\") "
Jul 24 04:04:03.633757 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633540    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-model-cache" (OuterVolumeSpecName: "model-cache") pod "21db5ab0-1de4-44b2-a926-6c18a6f2993a" (UID: "21db5ab0-1de4-44b2-a926-6c18a6f2993a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:04:03.633757 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633624    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:03.633757 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633623    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-home" (OuterVolumeSpecName: "home") pod "21db5ab0-1de4-44b2-a926-6c18a6f2993a" (UID: "21db5ab0-1de4-44b2-a926-6c18a6f2993a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:04:03.633757 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.633702    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "21db5ab0-1de4-44b2-a926-6c18a6f2993a" (UID: "21db5ab0-1de4-44b2-a926-6c18a6f2993a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:04:03.635758 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.635734    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "21db5ab0-1de4-44b2-a926-6c18a6f2993a" (UID: "21db5ab0-1de4-44b2-a926-6c18a6f2993a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:04:03.635856 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.635762    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-dshm" (OuterVolumeSpecName: "dshm") pod "21db5ab0-1de4-44b2-a926-6c18a6f2993a" (UID: "21db5ab0-1de4-44b2-a926-6c18a6f2993a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:04:03.635856 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.635827    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21db5ab0-1de4-44b2-a926-6c18a6f2993a-kube-api-access-r4p2c" (OuterVolumeSpecName: "kube-api-access-r4p2c") pod "21db5ab0-1de4-44b2-a926-6c18a6f2993a" (UID: "21db5ab0-1de4-44b2-a926-6c18a6f2993a"). InnerVolumeSpecName "kube-api-access-r4p2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:04:03.734600 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.734561    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:03.734600 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.734592    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:03.734600 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.734602    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:03.734600 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.734611    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21db5ab0-1de4-44b2-a926-6c18a6f2993a-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:03.734867 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.734619    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4p2c\" (UniqueName: \"kubernetes.io/projected/21db5ab0-1de4-44b2-a926-6c18a6f2993a-kube-api-access-r4p2c\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:03.798257 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.798223    2577 generic.go:356] "Generic (PLEG): container finished" podID="21db5ab0-1de4-44b2-a926-6c18a6f2993a" containerID="2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16" exitCode=0
Jul 24 04:04:03.798441 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.798281    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl" event={"ID":"21db5ab0-1de4-44b2-a926-6c18a6f2993a","Type":"ContainerDied","Data":"2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16"}
Jul 24 04:04:03.798441 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.798297    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"
Jul 24 04:04:03.798441 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.798315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl" event={"ID":"21db5ab0-1de4-44b2-a926-6c18a6f2993a","Type":"ContainerDied","Data":"8f1f1ac33e43b481553fcfb96772d7cef2eec96a2ea0ad9b7a43d1e5829b4f29"}
Jul 24 04:04:03.798441 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.798333    2577 scope.go:117] "RemoveContainer" containerID="2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16"
Jul 24 04:04:03.807349 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.807244    2577 scope.go:117] "RemoveContainer" containerID="2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16"
Jul 24 04:04:03.807568 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:04:03.807545    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16\": container with ID starting with 2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16 not found: ID does not exist" containerID="2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16"
Jul 24 04:04:03.807613 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.807578    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16"} err="failed to get container status \"2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16\": rpc error: code = NotFound desc = could not find container \"2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16\": container with ID starting with 2c5f6fa9232c0244f30ff40438e30ff19c9330de0eb4c6dd0c05f56fecf0ed16 not found: ID does not exist"
Jul 24 04:04:03.813644 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.813619    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"]
Jul 24 04:04:03.817179 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:03.817154    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/auth-test-model-5b56b9-kserve-759f67b77f-kc9nl"]
Jul 24 04:04:04.142416 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.142345    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"]
Jul 24 04:04:04.142696 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.142681    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21db5ab0-1de4-44b2-a926-6c18a6f2993a" containerName="main"
Jul 24 04:04:04.142739 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.142699    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="21db5ab0-1de4-44b2-a926-6c18a6f2993a" containerName="main"
Jul 24 04:04:04.142780 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.142771    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="21db5ab0-1de4-44b2-a926-6c18a6f2993a" containerName="main"
Jul 24 04:04:04.146819 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.146803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.149499 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.149473    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5b56b9\"/\"sub-test-model-5b56b9-kserve-self-signed-certs\""
Jul 24 04:04:04.149681 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.149661    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5b56b9\"/\"openshift-service-ca.crt\""
Jul 24 04:04:04.149737 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.149679    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5b56b9\"/\"default-dockercfg-6gjsc\""
Jul 24 04:04:04.149737 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.149691    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5b56b9\"/\"kube-root-ca.crt\""
Jul 24 04:04:04.155953 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.155927    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"]
Jul 24 04:04:04.156987 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.156956    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t" podUID="5538ba50-2405-485b-805e-1ec8a2c2a019" containerName="main" containerID="cri-o://558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213" gracePeriod=45
Jul 24 04:04:04.239066 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.239021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-dshm\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.239300 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.239082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-model-cache\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.239300 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.239107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f55afee4-e286-40d9-9829-e6e8796fbeeb-tls-certs\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.239300 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.239131    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-home\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.239300 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.239166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-tmp-dir\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.239300 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.239235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxnbl\" (UniqueName: \"kubernetes.io/projected/f55afee4-e286-40d9-9829-e6e8796fbeeb-kube-api-access-lxnbl\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.339973 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.339941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-model-cache\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.340436 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.339983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f55afee4-e286-40d9-9829-e6e8796fbeeb-tls-certs\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.340436 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.340011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-home\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.340436 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.340045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-tmp-dir\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.340436 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.340105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxnbl\" (UniqueName: \"kubernetes.io/projected/f55afee4-e286-40d9-9829-e6e8796fbeeb-kube-api-access-lxnbl\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.340436 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.340181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-dshm\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.340748 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.340500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-home\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.341373 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.341251    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-tmp-dir\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.341373 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.341305    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-model-cache\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.342838 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.342816    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-dshm\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.343842 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.343819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f55afee4-e286-40d9-9829-e6e8796fbeeb-tls-certs\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.349011 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.348987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxnbl\" (UniqueName: \"kubernetes.io/projected/f55afee4-e286-40d9-9829-e6e8796fbeeb-kube-api-access-lxnbl\") pod \"sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") " pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.415190 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.415159    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:04:04.440637 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.440608    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-model-cache\") pod \"5538ba50-2405-485b-805e-1ec8a2c2a019\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") "
Jul 24 04:04:04.440832 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.440664    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7ggv\" (UniqueName: \"kubernetes.io/projected/5538ba50-2405-485b-805e-1ec8a2c2a019-kube-api-access-h7ggv\") pod \"5538ba50-2405-485b-805e-1ec8a2c2a019\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") "
Jul 24 04:04:04.440832 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.440692    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-dshm\") pod \"5538ba50-2405-485b-805e-1ec8a2c2a019\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") "
Jul 24 04:04:04.440832 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.440724    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-home\") pod \"5538ba50-2405-485b-805e-1ec8a2c2a019\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") "
Jul 24 04:04:04.440832 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.440754    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-tmp-dir\") pod \"5538ba50-2405-485b-805e-1ec8a2c2a019\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") "
Jul 24 04:04:04.440832 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.440792    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5538ba50-2405-485b-805e-1ec8a2c2a019-tls-certs\") pod \"5538ba50-2405-485b-805e-1ec8a2c2a019\" (UID: \"5538ba50-2405-485b-805e-1ec8a2c2a019\") "
Jul 24 04:04:04.441097 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.440850    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-model-cache" (OuterVolumeSpecName: "model-cache") pod "5538ba50-2405-485b-805e-1ec8a2c2a019" (UID: "5538ba50-2405-485b-805e-1ec8a2c2a019"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:04:04.441097 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.441059    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5538ba50-2405-485b-805e-1ec8a2c2a019" (UID: "5538ba50-2405-485b-805e-1ec8a2c2a019"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:04:04.441097 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.441073    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-home" (OuterVolumeSpecName: "home") pod "5538ba50-2405-485b-805e-1ec8a2c2a019" (UID: "5538ba50-2405-485b-805e-1ec8a2c2a019"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:04:04.441097 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.441071    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:04.443190 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.443164    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-dshm" (OuterVolumeSpecName: "dshm") pod "5538ba50-2405-485b-805e-1ec8a2c2a019" (UID: "5538ba50-2405-485b-805e-1ec8a2c2a019"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:04:04.443331 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.443217    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5538ba50-2405-485b-805e-1ec8a2c2a019-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5538ba50-2405-485b-805e-1ec8a2c2a019" (UID: "5538ba50-2405-485b-805e-1ec8a2c2a019"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:04:04.443331 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.443313    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5538ba50-2405-485b-805e-1ec8a2c2a019-kube-api-access-h7ggv" (OuterVolumeSpecName: "kube-api-access-h7ggv") pod "5538ba50-2405-485b-805e-1ec8a2c2a019" (UID: "5538ba50-2405-485b-805e-1ec8a2c2a019"). InnerVolumeSpecName "kube-api-access-h7ggv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:04:04.457475 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.457450    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:04.542020 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.541975    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:04.542020 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.542014    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5538ba50-2405-485b-805e-1ec8a2c2a019-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:04.542020 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.542028    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7ggv\" (UniqueName: \"kubernetes.io/projected/5538ba50-2405-485b-805e-1ec8a2c2a019-kube-api-access-h7ggv\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:04.542374 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.542042    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:04.542374 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.542057    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5538ba50-2405-485b-805e-1ec8a2c2a019-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:04:04.593916 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.593888    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"]
Jul 24 04:04:04.595470 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:04:04.595441    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf55afee4_e286_40d9_9829_e6e8796fbeeb.slice/crio-f0d95aca3a8bc429d0bb71b5144fdb2e5137dc70815e7b6ee86e7185684e99a4 WatchSource:0}: Error finding container f0d95aca3a8bc429d0bb71b5144fdb2e5137dc70815e7b6ee86e7185684e99a4: Status 404 returned error can't find the container with id f0d95aca3a8bc429d0bb71b5144fdb2e5137dc70815e7b6ee86e7185684e99a4
Jul 24 04:04:04.805584 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.805547    2577 generic.go:356] "Generic (PLEG): container finished" podID="5538ba50-2405-485b-805e-1ec8a2c2a019" containerID="558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213" exitCode=0
Jul 24 04:04:04.805786 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.805594    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"
Jul 24 04:04:04.805786 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.805623    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t" event={"ID":"5538ba50-2405-485b-805e-1ec8a2c2a019","Type":"ContainerDied","Data":"558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213"}
Jul 24 04:04:04.805786 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.805664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t" event={"ID":"5538ba50-2405-485b-805e-1ec8a2c2a019","Type":"ContainerDied","Data":"ab31681eb196166fe4388a1b4416b480e7cd2fcf432b84a0fd42d1830f8853eb"}
Jul 24 04:04:04.805786 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.805687    2577 scope.go:117] "RemoveContainer" containerID="558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213"
Jul 24 04:04:04.807659 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.807632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc" event={"ID":"f55afee4-e286-40d9-9829-e6e8796fbeeb","Type":"ContainerStarted","Data":"b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342"}
Jul 24 04:04:04.807763 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.807677    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc" event={"ID":"f55afee4-e286-40d9-9829-e6e8796fbeeb","Type":"ContainerStarted","Data":"f0d95aca3a8bc429d0bb71b5144fdb2e5137dc70815e7b6ee86e7185684e99a4"}
Jul 24 04:04:04.815158 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.815134    2577 scope.go:117] "RemoveContainer" containerID="558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213"
Jul 24 04:04:04.815514 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:04:04.815493    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213\": container with ID starting with 558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213 not found: ID does not exist" containerID="558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213"
Jul 24 04:04:04.815575 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.815523    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213"} err="failed to get container status \"558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213\": rpc error: code = NotFound desc = could not find container \"558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213\": container with ID starting with 558a32aeb062a2769f9f3a2207153bb7f07fc1a8e36861c0643d371416671213 not found: ID does not exist"
Jul 24 04:04:04.825237 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.825186    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc" podStartSLOduration=0.825169715 podStartE2EDuration="825.169715ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:04:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:04:04.82456686 +0000 UTC m=+2811.585475342" watchObservedRunningTime="2026-07-24 04:04:04.825169715 +0000 UTC m=+2811.586078187"
Jul 24 04:04:04.840813 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.840773    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"]
Jul 24 04:04:04.845334 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:04.845302    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/auth-test-model-f701e0-kserve-55977d7f-p9k6t"]
Jul 24 04:04:05.644324 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:05.644230    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21db5ab0-1de4-44b2-a926-6c18a6f2993a" path="/var/lib/kubelet/pods/21db5ab0-1de4-44b2-a926-6c18a6f2993a/volumes"
Jul 24 04:04:05.644675 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:05.644634    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5538ba50-2405-485b-805e-1ec8a2c2a019" path="/var/lib/kubelet/pods/5538ba50-2405-485b-805e-1ec8a2c2a019/volumes"
Jul 24 04:04:13.887109 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:13.887069    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:04:14.458307 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:14.458247    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:14.458496 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:14.458326    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:14.470964 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:14.470928    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:14.879432 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:14.879335    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:04:25.730648 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.730612    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"]
Jul 24 04:04:25.731131 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.731103    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5538ba50-2405-485b-805e-1ec8a2c2a019" containerName="main"
Jul 24 04:04:25.731131 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.731120    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5538ba50-2405-485b-805e-1ec8a2c2a019" containerName="main"
Jul 24 04:04:25.731242 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.731204    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5538ba50-2405-485b-805e-1ec8a2c2a019" containerName="main"
Jul 24 04:04:25.737330 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.737308    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.741008 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.740981    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"]
Jul 24 04:04:25.741500 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.741476    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"kube-root-ca.crt\""
Jul 24 04:04:25.741623 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.741497    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"openshift-service-ca.crt\""
Jul 24 04:04:25.741623 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.741480    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"default-dockercfg-ttgsc\""
Jul 24 04:04:25.741623 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.741477    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"sub-test-model-f701e0-kserve-self-signed-certs\""
Jul 24 04:04:25.826224 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.826190    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2rks\" (UniqueName: \"kubernetes.io/projected/3693f535-9e47-4ada-bfcd-108f8cce80d1-kube-api-access-g2rks\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.826224 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.826239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-dshm\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.826470 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.826295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-home\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.826470 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.826395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-model-cache\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.826470 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.826428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-tmp-dir\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.826470 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.826443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3693f535-9e47-4ada-bfcd-108f8cce80d1-tls-certs\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.927064 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.927023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-home\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.927302 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.927074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-model-cache\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.927302 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.927093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-tmp-dir\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.927302 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.927133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3693f535-9e47-4ada-bfcd-108f8cce80d1-tls-certs\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.927302 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.927194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2rks\" (UniqueName: \"kubernetes.io/projected/3693f535-9e47-4ada-bfcd-108f8cce80d1-kube-api-access-g2rks\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.927302 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.927238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-dshm\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.927590 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.927482    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-home\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.927590 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.927551    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-model-cache\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.927590 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.927576    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-tmp-dir\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.929720 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.929695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-dshm\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.930017 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.930000    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3693f535-9e47-4ada-bfcd-108f8cce80d1-tls-certs\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:25.935644 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:25.935610    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2rks\" (UniqueName: \"kubernetes.io/projected/3693f535-9e47-4ada-bfcd-108f8cce80d1-kube-api-access-g2rks\") pod \"sub-test-model-f701e0-kserve-6dbd64578f-7l5ht\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") " pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:26.048349 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:26.048223    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:26.185804 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:26.185776    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"]
Jul 24 04:04:26.187822 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:04:26.187794    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3693f535_9e47_4ada_bfcd_108f8cce80d1.slice/crio-f41a76188e5fd149026fcfc72cf9b6a9b9a6b15e342bea50a5b8958b9d628410 WatchSource:0}: Error finding container f41a76188e5fd149026fcfc72cf9b6a9b9a6b15e342bea50a5b8958b9d628410: Status 404 returned error can't find the container with id f41a76188e5fd149026fcfc72cf9b6a9b9a6b15e342bea50a5b8958b9d628410
Jul 24 04:04:26.930375 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:26.930332    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht" event={"ID":"3693f535-9e47-4ada-bfcd-108f8cce80d1","Type":"ContainerStarted","Data":"77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f"}
Jul 24 04:04:26.930760 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:26.930382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht" event={"ID":"3693f535-9e47-4ada-bfcd-108f8cce80d1","Type":"ContainerStarted","Data":"f41a76188e5fd149026fcfc72cf9b6a9b9a6b15e342bea50a5b8958b9d628410"}
Jul 24 04:04:26.947423 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:26.947367    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht" podStartSLOduration=1.947350613 podStartE2EDuration="1.947350613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:04:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:04:26.947043663 +0000 UTC m=+2833.707952143" watchObservedRunningTime="2026-07-24 04:04:26.947350613 +0000 UTC m=+2833.708259093"
Jul 24 04:04:29.529254 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:29.529214    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:04:36.049486 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:36.049441    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:36.049987 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:36.049512    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:36.062259 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:36.062217    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:36.992626 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:36.992589    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:04:49.094119 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:04:49.094082    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:05:06.488386 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:06.488295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:05:13.264859 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:13.264819    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"]
Jul 24 04:05:14.168243 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:14.168209    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"]
Jul 24 04:05:20.618348 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.618312    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"]
Jul 24 04:05:20.621934 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.621916    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.624698 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.624674    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5b56b9\"/\"rate-test-model-5b56b9-kserve-self-signed-certs\""
Jul 24 04:05:20.631781 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.631757    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"]
Jul 24 04:05:20.716709 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.716669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tls-certs\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.716709 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.716707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-dshm\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.716950 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.716730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-home\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.716950 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.716793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9jb8\" (UniqueName: \"kubernetes.io/projected/ec6c2d98-45a3-4859-8045-25a0f741b9e9-kube-api-access-t9jb8\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.716950 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.716902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tmp-dir\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.717083 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.716957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-model-cache\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.817743 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.817699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-dshm\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.817959 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.817762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-home\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.817959 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.817804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9jb8\" (UniqueName: \"kubernetes.io/projected/ec6c2d98-45a3-4859-8045-25a0f741b9e9-kube-api-access-t9jb8\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.817959 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.817871    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tmp-dir\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.817959 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.817907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-model-cache\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.817959 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.817951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tls-certs\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.818235 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.818203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-home\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.818324 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.818234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tmp-dir\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.818391 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.818313    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-model-cache\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.820125 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.820098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-dshm\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.820690 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.820669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tls-certs\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.826238 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.826214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9jb8\" (UniqueName: \"kubernetes.io/projected/ec6c2d98-45a3-4859-8045-25a0f741b9e9-kube-api-access-t9jb8\") pod \"rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") " pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:20.932687 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:20.932650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:21.069589 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:21.069563    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"]
Jul 24 04:05:21.071565 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:05:21.071538    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec6c2d98_45a3_4859_8045_25a0f741b9e9.slice/crio-10fc892f991170cc75fecf59c70e7f63aa96e1bce7b15742660dbd8396f8b9b7 WatchSource:0}: Error finding container 10fc892f991170cc75fecf59c70e7f63aa96e1bce7b15742660dbd8396f8b9b7: Status 404 returned error can't find the container with id 10fc892f991170cc75fecf59c70e7f63aa96e1bce7b15742660dbd8396f8b9b7
Jul 24 04:05:21.215245 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:21.215144    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt" event={"ID":"ec6c2d98-45a3-4859-8045-25a0f741b9e9","Type":"ContainerStarted","Data":"357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e"}
Jul 24 04:05:21.215245 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:21.215198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt" event={"ID":"ec6c2d98-45a3-4859-8045-25a0f741b9e9","Type":"ContainerStarted","Data":"10fc892f991170cc75fecf59c70e7f63aa96e1bce7b15742660dbd8396f8b9b7"}
Jul 24 04:05:21.235300 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:21.235225    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt" podStartSLOduration=1.235207223 podStartE2EDuration="1.235207223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:05:21.232960497 +0000 UTC m=+2887.993868981" watchObservedRunningTime="2026-07-24 04:05:21.235207223 +0000 UTC m=+2887.996115703"
Jul 24 04:05:25.389212 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:25.389169    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:05:28.284725 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.284674    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc" podUID="f55afee4-e286-40d9-9829-e6e8796fbeeb" containerName="main" containerID="cri-o://b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342" gracePeriod=45
Jul 24 04:05:28.546421 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.546350    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:05:28.691915 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.691878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-dshm\") pod \"f55afee4-e286-40d9-9829-e6e8796fbeeb\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") "
Jul 24 04:05:28.691915 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.691933    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-model-cache\") pod \"f55afee4-e286-40d9-9829-e6e8796fbeeb\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") "
Jul 24 04:05:28.692206 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.691953    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f55afee4-e286-40d9-9829-e6e8796fbeeb-tls-certs\") pod \"f55afee4-e286-40d9-9829-e6e8796fbeeb\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") "
Jul 24 04:05:28.692206 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.691981    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-home\") pod \"f55afee4-e286-40d9-9829-e6e8796fbeeb\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") "
Jul 24 04:05:28.692206 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.692012    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxnbl\" (UniqueName: \"kubernetes.io/projected/f55afee4-e286-40d9-9829-e6e8796fbeeb-kube-api-access-lxnbl\") pod \"f55afee4-e286-40d9-9829-e6e8796fbeeb\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") "
Jul 24 04:05:28.692206 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.692137    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-tmp-dir\") pod \"f55afee4-e286-40d9-9829-e6e8796fbeeb\" (UID: \"f55afee4-e286-40d9-9829-e6e8796fbeeb\") "
Jul 24 04:05:28.692910 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.692852    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f55afee4-e286-40d9-9829-e6e8796fbeeb" (UID: "f55afee4-e286-40d9-9829-e6e8796fbeeb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:05:28.692910 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.692877    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-model-cache" (OuterVolumeSpecName: "model-cache") pod "f55afee4-e286-40d9-9829-e6e8796fbeeb" (UID: "f55afee4-e286-40d9-9829-e6e8796fbeeb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:05:28.693125 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.692922    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-home" (OuterVolumeSpecName: "home") pod "f55afee4-e286-40d9-9829-e6e8796fbeeb" (UID: "f55afee4-e286-40d9-9829-e6e8796fbeeb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:05:28.696049 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.695203    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f55afee4-e286-40d9-9829-e6e8796fbeeb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f55afee4-e286-40d9-9829-e6e8796fbeeb" (UID: "f55afee4-e286-40d9-9829-e6e8796fbeeb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:05:28.700523 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.700482    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-dshm" (OuterVolumeSpecName: "dshm") pod "f55afee4-e286-40d9-9829-e6e8796fbeeb" (UID: "f55afee4-e286-40d9-9829-e6e8796fbeeb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:05:28.700663 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.700646    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f55afee4-e286-40d9-9829-e6e8796fbeeb-kube-api-access-lxnbl" (OuterVolumeSpecName: "kube-api-access-lxnbl") pod "f55afee4-e286-40d9-9829-e6e8796fbeeb" (UID: "f55afee4-e286-40d9-9829-e6e8796fbeeb"). InnerVolumeSpecName "kube-api-access-lxnbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:05:28.793362 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.793327    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:28.793541 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.793374    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:28.793541 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.793388    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f55afee4-e286-40d9-9829-e6e8796fbeeb-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:28.793541 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.793396    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:28.793541 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.793405    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxnbl\" (UniqueName: \"kubernetes.io/projected/f55afee4-e286-40d9-9829-e6e8796fbeeb-kube-api-access-lxnbl\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:28.793541 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:28.793413    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f55afee4-e286-40d9-9829-e6e8796fbeeb-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:29.188148 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.188083    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht" podUID="3693f535-9e47-4ada-bfcd-108f8cce80d1" containerName="main" containerID="cri-o://77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f" gracePeriod=45
Jul 24 04:05:29.256414 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.256373    2577 generic.go:356] "Generic (PLEG): container finished" podID="f55afee4-e286-40d9-9829-e6e8796fbeeb" containerID="b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342" exitCode=0
Jul 24 04:05:29.256586 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.256441    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"
Jul 24 04:05:29.256586 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.256462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc" event={"ID":"f55afee4-e286-40d9-9829-e6e8796fbeeb","Type":"ContainerDied","Data":"b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342"}
Jul 24 04:05:29.256586 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.256509    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc" event={"ID":"f55afee4-e286-40d9-9829-e6e8796fbeeb","Type":"ContainerDied","Data":"f0d95aca3a8bc429d0bb71b5144fdb2e5137dc70815e7b6ee86e7185684e99a4"}
Jul 24 04:05:29.256586 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.256527    2577 scope.go:117] "RemoveContainer" containerID="b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342"
Jul 24 04:05:29.280037 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.280003    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"]
Jul 24 04:05:29.283902 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.283877    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/sub-test-model-5b56b9-kserve-78f99fd8d9-64ppc"]
Jul 24 04:05:29.303771 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.303654    2577 scope.go:117] "RemoveContainer" containerID="b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342"
Jul 24 04:05:29.304032 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:05:29.303993    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342\": container with ID starting with b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342 not found: ID does not exist" containerID="b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342"
Jul 24 04:05:29.304075 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.304030    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342"} err="failed to get container status \"b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342\": rpc error: code = NotFound desc = could not find container \"b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342\": container with ID starting with b6ccce185e7e48ee595918148e557433e99a06d699ba727d7fdb379b1f879342 not found: ID does not exist"
Jul 24 04:05:29.444659 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.444588    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:05:29.599317 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.599249    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-dshm\") pod \"3693f535-9e47-4ada-bfcd-108f8cce80d1\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") "
Jul 24 04:05:29.599507 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.599356    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2rks\" (UniqueName: \"kubernetes.io/projected/3693f535-9e47-4ada-bfcd-108f8cce80d1-kube-api-access-g2rks\") pod \"3693f535-9e47-4ada-bfcd-108f8cce80d1\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") "
Jul 24 04:05:29.599507 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.599406    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-home\") pod \"3693f535-9e47-4ada-bfcd-108f8cce80d1\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") "
Jul 24 04:05:29.599507 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.599447    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-tmp-dir\") pod \"3693f535-9e47-4ada-bfcd-108f8cce80d1\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") "
Jul 24 04:05:29.599507 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.599490    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-model-cache\") pod \"3693f535-9e47-4ada-bfcd-108f8cce80d1\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") "
Jul 24 04:05:29.599700 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.599526    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3693f535-9e47-4ada-bfcd-108f8cce80d1-tls-certs\") pod \"3693f535-9e47-4ada-bfcd-108f8cce80d1\" (UID: \"3693f535-9e47-4ada-bfcd-108f8cce80d1\") "
Jul 24 04:05:29.599758 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.599712    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-home" (OuterVolumeSpecName: "home") pod "3693f535-9e47-4ada-bfcd-108f8cce80d1" (UID: "3693f535-9e47-4ada-bfcd-108f8cce80d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:05:29.599758 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.599728    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3693f535-9e47-4ada-bfcd-108f8cce80d1" (UID: "3693f535-9e47-4ada-bfcd-108f8cce80d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:05:29.599866 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.599778    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "3693f535-9e47-4ada-bfcd-108f8cce80d1" (UID: "3693f535-9e47-4ada-bfcd-108f8cce80d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:05:29.601689 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.601659    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-dshm" (OuterVolumeSpecName: "dshm") pod "3693f535-9e47-4ada-bfcd-108f8cce80d1" (UID: "3693f535-9e47-4ada-bfcd-108f8cce80d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:05:29.601793 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.601685    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3693f535-9e47-4ada-bfcd-108f8cce80d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3693f535-9e47-4ada-bfcd-108f8cce80d1" (UID: "3693f535-9e47-4ada-bfcd-108f8cce80d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:05:29.601793 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.601737    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3693f535-9e47-4ada-bfcd-108f8cce80d1-kube-api-access-g2rks" (OuterVolumeSpecName: "kube-api-access-g2rks") pod "3693f535-9e47-4ada-bfcd-108f8cce80d1" (UID: "3693f535-9e47-4ada-bfcd-108f8cce80d1"). InnerVolumeSpecName "kube-api-access-g2rks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:05:29.642296 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.642246    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f55afee4-e286-40d9-9829-e6e8796fbeeb" path="/var/lib/kubelet/pods/f55afee4-e286-40d9-9829-e6e8796fbeeb/volumes"
Jul 24 04:05:29.700933 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.700839    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:29.700933 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.700867    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:29.700933 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.700886    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:29.700933 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.700894    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3693f535-9e47-4ada-bfcd-108f8cce80d1-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:29.700933 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.700901    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3693f535-9e47-4ada-bfcd-108f8cce80d1-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:29.700933 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:29.700909    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2rks\" (UniqueName: \"kubernetes.io/projected/3693f535-9e47-4ada-bfcd-108f8cce80d1-kube-api-access-g2rks\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:05:30.262845 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.262812    2577 generic.go:356] "Generic (PLEG): container finished" podID="3693f535-9e47-4ada-bfcd-108f8cce80d1" containerID="77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f" exitCode=0
Jul 24 04:05:30.263043 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.262857    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"
Jul 24 04:05:30.263043 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.262903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht" event={"ID":"3693f535-9e47-4ada-bfcd-108f8cce80d1","Type":"ContainerDied","Data":"77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f"}
Jul 24 04:05:30.263043 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.262943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht" event={"ID":"3693f535-9e47-4ada-bfcd-108f8cce80d1","Type":"ContainerDied","Data":"f41a76188e5fd149026fcfc72cf9b6a9b9a6b15e342bea50a5b8958b9d628410"}
Jul 24 04:05:30.263043 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.262963    2577 scope.go:117] "RemoveContainer" containerID="77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f"
Jul 24 04:05:30.272049 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.272028    2577 scope.go:117] "RemoveContainer" containerID="77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f"
Jul 24 04:05:30.272349 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:05:30.272328    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f\": container with ID starting with 77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f not found: ID does not exist" containerID="77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f"
Jul 24 04:05:30.272432 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.272358    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f"} err="failed to get container status \"77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f\": rpc error: code = NotFound desc = could not find container \"77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f\": container with ID starting with 77c88a498f2555403278bfb129a712db18f08f8e7345595fbbf5616cc7dd6c2f not found: ID does not exist"
Jul 24 04:05:30.279205 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.279164    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"]
Jul 24 04:05:30.281437 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.281413    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/sub-test-model-f701e0-kserve-6dbd64578f-7l5ht"]
Jul 24 04:05:30.933242 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.933205    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:30.933660 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.933284    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:30.945665 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:30.945636    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:31.281730 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:31.281649    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:05:31.642979 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:31.642877    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3693f535-9e47-4ada-bfcd-108f8cce80d1" path="/var/lib/kubelet/pods/3693f535-9e47-4ada-bfcd-108f8cce80d1/volumes"
Jul 24 04:05:42.432790 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.432756    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"]
Jul 24 04:05:42.433340 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.433104    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f55afee4-e286-40d9-9829-e6e8796fbeeb" containerName="main"
Jul 24 04:05:42.433340 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.433115    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f55afee4-e286-40d9-9829-e6e8796fbeeb" containerName="main"
Jul 24 04:05:42.433340 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.433125    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3693f535-9e47-4ada-bfcd-108f8cce80d1" containerName="main"
Jul 24 04:05:42.433340 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.433130    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3693f535-9e47-4ada-bfcd-108f8cce80d1" containerName="main"
Jul 24 04:05:42.433340 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.433203    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3693f535-9e47-4ada-bfcd-108f8cce80d1" containerName="main"
Jul 24 04:05:42.433340 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.433213    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f55afee4-e286-40d9-9829-e6e8796fbeeb" containerName="main"
Jul 24 04:05:42.437435 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.437411    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.440910 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.440882    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"default-dockercfg-ttgsc\""
Jul 24 04:05:42.441057 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.440911    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"openshift-service-ca.crt\""
Jul 24 04:05:42.441057 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.440943    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"rate-test-model-f701e0-kserve-self-signed-certs\""
Jul 24 04:05:42.441057 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.440884    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f701e0\"/\"kube-root-ca.crt\""
Jul 24 04:05:42.446510 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.446485    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"]
Jul 24 04:05:42.511648 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.511609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tmp-dir\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.511817 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.511682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjgnf\" (UniqueName: \"kubernetes.io/projected/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-kube-api-access-gjgnf\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.511817 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.511708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-home\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.511817 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.511724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-model-cache\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.511817 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.511754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-dshm\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.511817 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.511776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tls-certs\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.613003 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.612961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-dshm\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.613003 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.613001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tls-certs\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.613302 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.613065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tmp-dir\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.613302 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.613138    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjgnf\" (UniqueName: \"kubernetes.io/projected/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-kube-api-access-gjgnf\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.613302 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.613164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-home\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.613302 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.613184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-model-cache\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.613553 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.613533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-model-cache\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.613610 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.613577    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-home\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.613664 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.613646    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tmp-dir\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.616022 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.616001    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-dshm\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.616412 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.616391    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tls-certs\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.621220 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.621198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjgnf\" (UniqueName: \"kubernetes.io/projected/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-kube-api-access-gjgnf\") pod \"rate-test-model-f701e0-kserve-65dbb7f587-tdkfc\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") " pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.747800 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.747699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:42.888565 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:42.888538    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"]
Jul 24 04:05:42.890536 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:05:42.890507    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88aeb027_c5c9_42b6_8ea0_5881e5c94d8a.slice/crio-38489f6343db4693971d6ebea5dd6a1b4261e8d8506c6a077ce69d7ea319beb6 WatchSource:0}: Error finding container 38489f6343db4693971d6ebea5dd6a1b4261e8d8506c6a077ce69d7ea319beb6: Status 404 returned error can't find the container with id 38489f6343db4693971d6ebea5dd6a1b4261e8d8506c6a077ce69d7ea319beb6
Jul 24 04:05:43.345569 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:43.345530    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc" event={"ID":"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a","Type":"ContainerStarted","Data":"ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad"}
Jul 24 04:05:43.345569 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:43.345573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc" event={"ID":"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a","Type":"ContainerStarted","Data":"38489f6343db4693971d6ebea5dd6a1b4261e8d8506c6a077ce69d7ea319beb6"}
Jul 24 04:05:43.361853 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:43.361802    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc" podStartSLOduration=1.361787457 podStartE2EDuration="1.361787457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:05:43.361299795 +0000 UTC m=+2910.122208272" watchObservedRunningTime="2026-07-24 04:05:43.361787457 +0000 UTC m=+2910.122695915"
Jul 24 04:05:45.990800 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:45.990760    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:05:52.748748 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:52.748705    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:52.748748 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:52.748757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:52.761743 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:52.761711    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:05:53.417572 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:05:53.417533    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:06:03.400428 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:03.400383    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:06:18.986385 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:18.986334    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:06:25.213025 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:25.212991    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"]
Jul 24 04:06:26.219398 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:26.219365    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"]
Jul 24 04:06:35.869383 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:35.869300    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7627-140150868692832"]
Jul 24 04:06:35.872734 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:35.872715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7627-140150868692832"
Jul 24 04:06:35.875769 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:35.875741    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-sqpgk\""
Jul 24 04:06:35.878109 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:35.878080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vqbz\" (UniqueName: \"kubernetes.io/projected/9627c7ba-164e-47fa-bdb2-9343eac67539-kube-api-access-6vqbz\") pod \"test-curl-7627-140150868692832\" (UID: \"9627c7ba-164e-47fa-bdb2-9343eac67539\") " pod="opendatahub/test-curl-7627-140150868692832"
Jul 24 04:06:35.878361 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:35.878341    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7627-140150868692832"]
Jul 24 04:06:35.978810 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:35.978772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vqbz\" (UniqueName: \"kubernetes.io/projected/9627c7ba-164e-47fa-bdb2-9343eac67539-kube-api-access-6vqbz\") pod \"test-curl-7627-140150868692832\" (UID: \"9627c7ba-164e-47fa-bdb2-9343eac67539\") " pod="opendatahub/test-curl-7627-140150868692832"
Jul 24 04:06:35.987366 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:35.987343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vqbz\" (UniqueName: \"kubernetes.io/projected/9627c7ba-164e-47fa-bdb2-9343eac67539-kube-api-access-6vqbz\") pod \"test-curl-7627-140150868692832\" (UID: \"9627c7ba-164e-47fa-bdb2-9343eac67539\") " pod="opendatahub/test-curl-7627-140150868692832"
Jul 24 04:06:36.183545 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:36.183511    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7627-140150868692832"
Jul 24 04:06:36.324965 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:36.324933    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7627-140150868692832"]
Jul 24 04:06:36.326639 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:06:36.326610    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9627c7ba_164e_47fa_bdb2_9343eac67539.slice/crio-a564088d7415d84140161538ffed9f31cebe864e81ec47e2c41206f4842117f5 WatchSource:0}: Error finding container a564088d7415d84140161538ffed9f31cebe864e81ec47e2c41206f4842117f5: Status 404 returned error can't find the container with id a564088d7415d84140161538ffed9f31cebe864e81ec47e2c41206f4842117f5
Jul 24 04:06:36.327952 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:36.327929    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 04:06:36.635013 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:36.634925    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7627-140150868692832" event={"ID":"9627c7ba-164e-47fa-bdb2-9343eac67539","Type":"ContainerStarted","Data":"a564088d7415d84140161538ffed9f31cebe864e81ec47e2c41206f4842117f5"}
Jul 24 04:06:37.643035 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:37.642998    2577 generic.go:356] "Generic (PLEG): container finished" podID="9627c7ba-164e-47fa-bdb2-9343eac67539" containerID="a59b1a0506bf4131406089c0a8a113410bb6ffaa0d1e4e02fae8ba044455910b" exitCode=0
Jul 24 04:06:37.643035 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:37.643035    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7627-140150868692832" event={"ID":"9627c7ba-164e-47fa-bdb2-9343eac67539","Type":"ContainerDied","Data":"a59b1a0506bf4131406089c0a8a113410bb6ffaa0d1e4e02fae8ba044455910b"}
Jul 24 04:06:37.689976 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:37.689941    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:06:38.776380 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:38.776351    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7627-140150868692832"
Jul 24 04:06:38.801493 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:38.801464    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vqbz\" (UniqueName: \"kubernetes.io/projected/9627c7ba-164e-47fa-bdb2-9343eac67539-kube-api-access-6vqbz\") pod \"9627c7ba-164e-47fa-bdb2-9343eac67539\" (UID: \"9627c7ba-164e-47fa-bdb2-9343eac67539\") "
Jul 24 04:06:38.803949 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:38.803918    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9627c7ba-164e-47fa-bdb2-9343eac67539-kube-api-access-6vqbz" (OuterVolumeSpecName: "kube-api-access-6vqbz") pod "9627c7ba-164e-47fa-bdb2-9343eac67539" (UID: "9627c7ba-164e-47fa-bdb2-9343eac67539"). InnerVolumeSpecName "kube-api-access-6vqbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:06:38.820992 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:38.820966    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7627-140150868692832_9627c7ba-164e-47fa-bdb2-9343eac67539/test-curl-7627-140150868692832/0.log"
Jul 24 04:06:38.848891 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:38.848843    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7627-140150868692832"]
Jul 24 04:06:38.852944 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:38.852915    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7627-140150868692832"]
Jul 24 04:06:38.902381 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:38.902300    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vqbz\" (UniqueName: \"kubernetes.io/projected/9627c7ba-164e-47fa-bdb2-9343eac67539-kube-api-access-6vqbz\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:39.645698 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:39.645663    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9627c7ba-164e-47fa-bdb2-9343eac67539" path="/var/lib/kubelet/pods/9627c7ba-164e-47fa-bdb2-9343eac67539/volumes"
Jul 24 04:06:39.661047 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:39.661014    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7627-140150868692832"
Jul 24 04:06:39.661209 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:39.661018    2577 scope.go:117] "RemoveContainer" containerID="a59b1a0506bf4131406089c0a8a113410bb6ffaa0d1e4e02fae8ba044455910b"
Jul 24 04:06:40.232652 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.232612    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt" podUID="ec6c2d98-45a3-4859-8045-25a0f741b9e9" containerName="main" containerID="cri-o://357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e" gracePeriod=45
Jul 24 04:06:40.477294 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.477252    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:06:40.517837 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.517759    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tls-certs\") pod \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") "
Jul 24 04:06:40.517837 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.517809    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9jb8\" (UniqueName: \"kubernetes.io/projected/ec6c2d98-45a3-4859-8045-25a0f741b9e9-kube-api-access-t9jb8\") pod \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") "
Jul 24 04:06:40.517837 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.517830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tmp-dir\") pod \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") "
Jul 24 04:06:40.518131 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.517855    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-model-cache\") pod \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") "
Jul 24 04:06:40.518131 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.517874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-home\") pod \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") "
Jul 24 04:06:40.518131 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.517911    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-dshm\") pod \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\" (UID: \"ec6c2d98-45a3-4859-8045-25a0f741b9e9\") "
Jul 24 04:06:40.518309 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.518121    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-home" (OuterVolumeSpecName: "home") pod "ec6c2d98-45a3-4859-8045-25a0f741b9e9" (UID: "ec6c2d98-45a3-4859-8045-25a0f741b9e9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:06:40.518309 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.518132    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-model-cache" (OuterVolumeSpecName: "model-cache") pod "ec6c2d98-45a3-4859-8045-25a0f741b9e9" (UID: "ec6c2d98-45a3-4859-8045-25a0f741b9e9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:06:40.518309 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.518141    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ec6c2d98-45a3-4859-8045-25a0f741b9e9" (UID: "ec6c2d98-45a3-4859-8045-25a0f741b9e9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:06:40.518309 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.518242    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:40.518309 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.518256    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:40.518309 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.518284    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:40.520121 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.520098    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-dshm" (OuterVolumeSpecName: "dshm") pod "ec6c2d98-45a3-4859-8045-25a0f741b9e9" (UID: "ec6c2d98-45a3-4859-8045-25a0f741b9e9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:06:40.520201 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.520170    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ec6c2d98-45a3-4859-8045-25a0f741b9e9" (UID: "ec6c2d98-45a3-4859-8045-25a0f741b9e9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:06:40.520201 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.520187    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec6c2d98-45a3-4859-8045-25a0f741b9e9-kube-api-access-t9jb8" (OuterVolumeSpecName: "kube-api-access-t9jb8") pod "ec6c2d98-45a3-4859-8045-25a0f741b9e9" (UID: "ec6c2d98-45a3-4859-8045-25a0f741b9e9"). InnerVolumeSpecName "kube-api-access-t9jb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:06:40.618809 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.618754    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec6c2d98-45a3-4859-8045-25a0f741b9e9-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:40.618809 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.618801    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9jb8\" (UniqueName: \"kubernetes.io/projected/ec6c2d98-45a3-4859-8045-25a0f741b9e9-kube-api-access-t9jb8\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:40.618809 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.618815    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec6c2d98-45a3-4859-8045-25a0f741b9e9-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:40.667724 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.667688    2577 generic.go:356] "Generic (PLEG): container finished" podID="ec6c2d98-45a3-4859-8045-25a0f741b9e9" containerID="357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e" exitCode=0
Jul 24 04:06:40.667894 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.667743    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"
Jul 24 04:06:40.667894 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.667773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt" event={"ID":"ec6c2d98-45a3-4859-8045-25a0f741b9e9","Type":"ContainerDied","Data":"357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e"}
Jul 24 04:06:40.667894 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.667822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt" event={"ID":"ec6c2d98-45a3-4859-8045-25a0f741b9e9","Type":"ContainerDied","Data":"10fc892f991170cc75fecf59c70e7f63aa96e1bce7b15742660dbd8396f8b9b7"}
Jul 24 04:06:40.667894 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.667846    2577 scope.go:117] "RemoveContainer" containerID="357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e"
Jul 24 04:06:40.677499 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.677476    2577 scope.go:117] "RemoveContainer" containerID="357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e"
Jul 24 04:06:40.677743 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:06:40.677723    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e\": container with ID starting with 357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e not found: ID does not exist" containerID="357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e"
Jul 24 04:06:40.677818 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.677752    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e"} err="failed to get container status \"357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e\": rpc error: code = NotFound desc = could not find container \"357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e\": container with ID starting with 357a8adb866dc3d8e4caa2ceb2ec05f2653db8a0669fe9940f30ca36ba61e34e not found: ID does not exist"
Jul 24 04:06:40.689248 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.689222    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"]
Jul 24 04:06:40.693444 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:40.693421    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5b56b9/rate-test-model-5b56b9-kserve-6f4559bd87-bnwnt"]
Jul 24 04:06:41.238733 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.238672    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc" podUID="88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" containerName="main" containerID="cri-o://ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad" gracePeriod=45
Jul 24 04:06:41.490008 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.489943    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:06:41.526514 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.526482    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjgnf\" (UniqueName: \"kubernetes.io/projected/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-kube-api-access-gjgnf\") pod \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") "
Jul 24 04:06:41.526688 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.526529    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-dshm\") pod \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") "
Jul 24 04:06:41.526688 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.526571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-home\") pod \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") "
Jul 24 04:06:41.526688 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.526588    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tls-certs\") pod \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") "
Jul 24 04:06:41.526688 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.526624    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tmp-dir\") pod \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") "
Jul 24 04:06:41.526688 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.526647    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-model-cache\") pod \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\" (UID: \"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a\") "
Jul 24 04:06:41.526975 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.526948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-home" (OuterVolumeSpecName: "home") pod "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" (UID: "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:06:41.526975 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.526963    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" (UID: "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:06:41.527099 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.527079    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-model-cache" (OuterVolumeSpecName: "model-cache") pod "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" (UID: "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:06:41.529103 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.529077    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" (UID: "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:06:41.529250 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.529234    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-dshm" (OuterVolumeSpecName: "dshm") pod "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" (UID: "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:06:41.529345 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.529315    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-kube-api-access-gjgnf" (OuterVolumeSpecName: "kube-api-access-gjgnf") pod "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" (UID: "88aeb027-c5c9-42b6-8ea0-5881e5c94d8a"). InnerVolumeSpecName "kube-api-access-gjgnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:06:41.627638 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.627598    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:41.627638 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.627630    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:41.627638 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.627641    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjgnf\" (UniqueName: \"kubernetes.io/projected/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-kube-api-access-gjgnf\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:41.627888 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.627650    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:41.627888 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.627659    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:41.627888 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.627666    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:41.643061 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.643028    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec6c2d98-45a3-4859-8045-25a0f741b9e9" path="/var/lib/kubelet/pods/ec6c2d98-45a3-4859-8045-25a0f741b9e9/volumes"
Jul 24 04:06:41.676462 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.676432    2577 generic.go:356] "Generic (PLEG): container finished" podID="88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" containerID="ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad" exitCode=0
Jul 24 04:06:41.676657 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.676475    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"
Jul 24 04:06:41.676657 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.676484    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc" event={"ID":"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a","Type":"ContainerDied","Data":"ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad"}
Jul 24 04:06:41.676657 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.676516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc" event={"ID":"88aeb027-c5c9-42b6-8ea0-5881e5c94d8a","Type":"ContainerDied","Data":"38489f6343db4693971d6ebea5dd6a1b4261e8d8506c6a077ce69d7ea319beb6"}
Jul 24 04:06:41.676657 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.676534    2577 scope.go:117] "RemoveContainer" containerID="ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad"
Jul 24 04:06:41.685489 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.685466    2577 scope.go:117] "RemoveContainer" containerID="ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad"
Jul 24 04:06:41.685790 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:06:41.685768    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad\": container with ID starting with ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad not found: ID does not exist" containerID="ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad"
Jul 24 04:06:41.685859 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.685802    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad"} err="failed to get container status \"ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad\": rpc error: code = NotFound desc = could not find container \"ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad\": container with ID starting with ba65030311e0faa459e954d5c65b3dc7a05b85129abe6f39fa164bef6c7539ad not found: ID does not exist"
Jul 24 04:06:41.696203 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.696175    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"]
Jul 24 04:06:41.700563 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.700542    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f701e0/rate-test-model-f701e0-kserve-65dbb7f587-tdkfc"]
Jul 24 04:06:41.910761 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.910733    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7627-140150871897552"]
Jul 24 04:06:41.911123 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.911106    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" containerName="main"
Jul 24 04:06:41.911200 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.911126    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" containerName="main"
Jul 24 04:06:41.911200 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.911140    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9627c7ba-164e-47fa-bdb2-9343eac67539" containerName="test-curl-7627-140150868692832"
Jul 24 04:06:41.911200 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.911153    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9627c7ba-164e-47fa-bdb2-9343eac67539" containerName="test-curl-7627-140150868692832"
Jul 24 04:06:41.911200 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.911181    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec6c2d98-45a3-4859-8045-25a0f741b9e9" containerName="main"
Jul 24 04:06:41.911200 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.911190    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec6c2d98-45a3-4859-8045-25a0f741b9e9" containerName="main"
Jul 24 04:06:41.911515 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.911291    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" containerName="main"
Jul 24 04:06:41.911515 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.911305    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9627c7ba-164e-47fa-bdb2-9343eac67539" containerName="test-curl-7627-140150868692832"
Jul 24 04:06:41.911515 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.911313    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec6c2d98-45a3-4859-8045-25a0f741b9e9" containerName="main"
Jul 24 04:06:41.915496 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.915476    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7627-140150871897552"
Jul 24 04:06:41.918311 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.918291    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-sqpgk\""
Jul 24 04:06:41.921812 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.921786    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7627-140150871897552"]
Jul 24 04:06:41.931390 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:41.931364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9vn9\" (UniqueName: \"kubernetes.io/projected/88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484-kube-api-access-h9vn9\") pod \"test-curl-7627-140150871897552\" (UID: \"88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484\") " pod="opendatahub/test-curl-7627-140150871897552"
Jul 24 04:06:42.032156 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:42.032113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9vn9\" (UniqueName: \"kubernetes.io/projected/88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484-kube-api-access-h9vn9\") pod \"test-curl-7627-140150871897552\" (UID: \"88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484\") " pod="opendatahub/test-curl-7627-140150871897552"
Jul 24 04:06:42.041358 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:42.041332    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9vn9\" (UniqueName: \"kubernetes.io/projected/88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484-kube-api-access-h9vn9\") pod \"test-curl-7627-140150871897552\" (UID: \"88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484\") " pod="opendatahub/test-curl-7627-140150871897552"
Jul 24 04:06:42.226084 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:42.225986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7627-140150871897552"
Jul 24 04:06:42.363199 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:42.363171    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7627-140150871897552"]
Jul 24 04:06:42.365744 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:06:42.365713    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88ee44de_f4e9_4b9f_9e7a_9c55ee8c5484.slice/crio-e3003c8c98f249b562bcdd13698c255aba08f09d73bc55bbf93b12c99d46e372 WatchSource:0}: Error finding container e3003c8c98f249b562bcdd13698c255aba08f09d73bc55bbf93b12c99d46e372: Status 404 returned error can't find the container with id e3003c8c98f249b562bcdd13698c255aba08f09d73bc55bbf93b12c99d46e372
Jul 24 04:06:42.684154 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:42.684119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7627-140150871897552" event={"ID":"88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484","Type":"ContainerStarted","Data":"e3003c8c98f249b562bcdd13698c255aba08f09d73bc55bbf93b12c99d46e372"}
Jul 24 04:06:43.642243 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:43.642196    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88aeb027-c5c9-42b6-8ea0-5881e5c94d8a" path="/var/lib/kubelet/pods/88aeb027-c5c9-42b6-8ea0-5881e5c94d8a/volumes"
Jul 24 04:06:43.690249 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:43.690207    2577 generic.go:356] "Generic (PLEG): container finished" podID="88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484" containerID="c98f997b07c8981e40288a1bc6bfd101d74fccd33e1d4518842aeac80967c6f8" exitCode=0
Jul 24 04:06:43.690457 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:43.690296    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7627-140150871897552" event={"ID":"88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484","Type":"ContainerDied","Data":"c98f997b07c8981e40288a1bc6bfd101d74fccd33e1d4518842aeac80967c6f8"}
Jul 24 04:06:44.824893 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:44.824857    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7627-140150871897552"
Jul 24 04:06:44.855954 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:44.855920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9vn9\" (UniqueName: \"kubernetes.io/projected/88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484-kube-api-access-h9vn9\") pod \"88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484\" (UID: \"88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484\") "
Jul 24 04:06:44.858187 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:44.858159    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484-kube-api-access-h9vn9" (OuterVolumeSpecName: "kube-api-access-h9vn9") pod "88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484" (UID: "88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484"). InnerVolumeSpecName "kube-api-access-h9vn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:06:44.885353 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:44.885326    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7627-140150871897552_88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484/test-curl-7627-140150871897552/0.log"
Jul 24 04:06:44.933723 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:44.933689    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7627-140150871897552"]
Jul 24 04:06:44.940823 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:44.940794    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7627-140150871897552"]
Jul 24 04:06:44.957475 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:44.957411    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9vn9\" (UniqueName: \"kubernetes.io/projected/88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484-kube-api-access-h9vn9\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:06:45.642955 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:45.642921    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484" path="/var/lib/kubelet/pods/88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484/volumes"
Jul 24 04:06:45.701988 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:45.701956    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7627-140150871897552"
Jul 24 04:06:45.701988 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:45.701985    2577 scope.go:117] "RemoveContainer" containerID="c98f997b07c8981e40288a1bc6bfd101d74fccd33e1d4518842aeac80967c6f8"
Jul 24 04:06:48.758008 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.757964    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"]
Jul 24 04:06:48.758398 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.758348    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484" containerName="test-curl-7627-140150871897552"
Jul 24 04:06:48.758398 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.758362    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484" containerName="test-curl-7627-140150871897552"
Jul 24 04:06:48.758490 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.758434    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88ee44de-f4e9-4b9f-9e7a-9c55ee8c5484" containerName="test-curl-7627-140150871897552"
Jul 24 04:06:48.762677 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.762660    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:48.765375 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.765358    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-733dde-serving-cert\""
Jul 24 04:06:48.776650 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.776627    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"]
Jul 24 04:06:48.794564 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.794536    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:48.794700 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.794568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vkw6\" (UniqueName: \"kubernetes.io/projected/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-kube-api-access-8vkw6\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:48.794700 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.794616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:48.895310 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.895248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:48.895487 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.895367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:48.895487 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.895388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vkw6\" (UniqueName: \"kubernetes.io/projected/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-kube-api-access-8vkw6\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:48.895565 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:06:48.895528    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-serving-cert: secret "maas-api-e2e-ipp-a-733dde-serving-cert" not found
Jul 24 04:06:48.895623 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:06:48.895610    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-maas-api-tls podName:dbc3d6fb-5fc4-4cc7-9f70-ca288750641e nodeName:}" failed. No retries permitted until 2026-07-24 04:06:49.395587635 +0000 UTC m=+2976.156496111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-maas-api-tls") pod "maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x" (UID: "dbc3d6fb-5fc4-4cc7-9f70-ca288750641e") : secret "maas-api-e2e-ipp-a-733dde-serving-cert" not found
Jul 24 04:06:48.896023 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.896002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:48.907989 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:48.907958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vkw6\" (UniqueName: \"kubernetes.io/projected/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-kube-api-access-8vkw6\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:49.401397 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:49.401361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:49.403980 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:49.403958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:49.672968 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:49.672940    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:49.849190 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:49.842771    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"]
Jul 24 04:06:50.729286 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:50.729239    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x" event={"ID":"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e","Type":"ContainerStarted","Data":"f2561553b32c802f79e7a83ffcefda9c8eb472c8a0b48cf43f91ef088503aef4"}
Jul 24 04:06:50.729510 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:50.729322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x" event={"ID":"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e","Type":"ContainerStarted","Data":"0a3bea5be9e556b295ebf66522b77de8ce76807ad85789598bef07d69bef317e"}
Jul 24 04:06:50.729510 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:50.729376    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:06:50.750865 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:50.750802    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x" podStartSLOduration=2.278946823 podStartE2EDuration="2.750786557s" totalImagesPullingTime="471.839734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:06:50.748711463 +0000 UTC m=+2977.509619945" watchObservedRunningTime="2026-07-24 04:06:50.750786557 +0000 UTC m=+2977.511695037"
Jul 24 04:06:54.494507 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:54.494469    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:06:56.739727 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:06:56.739693    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:07:07.854676 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:07.854636    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"]
Jul 24 04:07:07.858118 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:07.858092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:07.860802 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:07.860778    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-0cae5e-serving-cert\""
Jul 24 04:07:07.872951 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:07.872917    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"]
Jul 24 04:07:07.958065 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:07.958031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgznv\" (UniqueName: \"kubernetes.io/projected/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-kube-api-access-cgznv\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:07.958254 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:07.958101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:07.958254 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:07.958121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:08.058690 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:08.058648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgznv\" (UniqueName: \"kubernetes.io/projected/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-kube-api-access-cgznv\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:08.058894 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:08.058711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:08.058894 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:08.058727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:08.058894 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:07:08.058840    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-serving-cert: secret "maas-api-e2e-ipp-b-0cae5e-serving-cert" not found
Jul 24 04:07:08.059066 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:07:08.058932    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-maas-api-tls podName:25d14fd0-b7b4-484c-a1f0-3669f872d8f0 nodeName:}" failed. No retries permitted until 2026-07-24 04:07:08.558911016 +0000 UTC m=+2995.319819481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-maas-api-tls") pod "maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw" (UID: "25d14fd0-b7b4-484c-a1f0-3669f872d8f0") : secret "maas-api-e2e-ipp-b-0cae5e-serving-cert" not found
Jul 24 04:07:08.059485 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:08.059468    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:08.073509 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:08.073485    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgznv\" (UniqueName: \"kubernetes.io/projected/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-kube-api-access-cgznv\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:08.562941 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:08.562895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:08.565533 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:08.565510    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:08.768665 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:08.768624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:08.917796 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:08.917768    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"]
Jul 24 04:07:08.918941 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:07:08.918914    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25d14fd0_b7b4_484c_a1f0_3669f872d8f0.slice/crio-3ab9c21aad445b7f667e5d3e3fc6c1e898a90a0c1e3fe441e1ab098b17461a4a WatchSource:0}: Error finding container 3ab9c21aad445b7f667e5d3e3fc6c1e898a90a0c1e3fe441e1ab098b17461a4a: Status 404 returned error can't find the container with id 3ab9c21aad445b7f667e5d3e3fc6c1e898a90a0c1e3fe441e1ab098b17461a4a
Jul 24 04:07:09.835078 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:09.835038    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw" event={"ID":"25d14fd0-b7b4-484c-a1f0-3669f872d8f0","Type":"ContainerStarted","Data":"3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b"}
Jul 24 04:07:09.835259 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:09.835096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw" event={"ID":"25d14fd0-b7b4-484c-a1f0-3669f872d8f0","Type":"ContainerStarted","Data":"3ab9c21aad445b7f667e5d3e3fc6c1e898a90a0c1e3fe441e1ab098b17461a4a"}
Jul 24 04:07:09.835259 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:09.835148    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:09.851856 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:09.851805    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw" podStartSLOduration=2.029292812 podStartE2EDuration="2.851791608s" totalImagesPullingTime="822.498796ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:07:09.850716869 +0000 UTC m=+2996.611625350" watchObservedRunningTime="2026-07-24 04:07:09.851791608 +0000 UTC m=+2996.612700087"
Jul 24 04:07:13.979237 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:13.979201    2577 scope.go:117] "RemoveContainer" containerID="c7d68eabdc0bd9d0ea7728c9cb1dc84c1c91d8d93f29576a99bd10617a6fb60c"
Jul 24 04:07:15.846280 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:15.846235    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:07:17.783766 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:17.783727    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:07:36.944192 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:36.944148    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"]
Jul 24 04:07:36.947586 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:36.947568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:36.950058 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:36.950032    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-733dde\"/\"openshift-service-ca.crt\""
Jul 24 04:07:36.950985 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:36.950968    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-733dde\"/\"default-dockercfg-5xmpj\""
Jul 24 04:07:36.951076 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:36.951029    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-733dde\"/\"ipp-route-733dde-kserve-self-signed-certs\""
Jul 24 04:07:36.951076 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:36.951049    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-733dde\"/\"kube-root-ca.crt\""
Jul 24 04:07:36.961867 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:36.961839    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"]
Jul 24 04:07:37.018483 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.018453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-dshm\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.018483 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.018489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-home\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.018700 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.018519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-tmp-dir\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.018700 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.018540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2abf2f1b-9975-4e5e-9f43-a10c30353462-tls-certs\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.018700 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.018573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d5rc\" (UniqueName: \"kubernetes.io/projected/2abf2f1b-9975-4e5e-9f43-a10c30353462-kube-api-access-7d5rc\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.018700 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.018607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-model-cache\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.119874 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.119837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-dshm\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.119874 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.119881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-home\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.120140 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.119911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-tmp-dir\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.120140 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.119930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2abf2f1b-9975-4e5e-9f43-a10c30353462-tls-certs\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.120140 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.120047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d5rc\" (UniqueName: \"kubernetes.io/projected/2abf2f1b-9975-4e5e-9f43-a10c30353462-kube-api-access-7d5rc\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.120140 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.120102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-model-cache\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.120400 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.120327    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-home\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.120459 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.120401    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-tmp-dir\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.120497 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.120466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-model-cache\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.122480 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.122456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-dshm\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.122753 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.122738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2abf2f1b-9975-4e5e-9f43-a10c30353462-tls-certs\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.131946 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.131920    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d5rc\" (UniqueName: \"kubernetes.io/projected/2abf2f1b-9975-4e5e-9f43-a10c30353462-kube-api-access-7d5rc\") pod \"ipp-route-733dde-kserve-5f4d48fbbc-5k49w\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") " pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.256993 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.256899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:37.389712 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.389636    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"]
Jul 24 04:07:37.391749 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:07:37.391713    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2abf2f1b_9975_4e5e_9f43_a10c30353462.slice/crio-89962f5dfe9d58bda6e2106e5a5d9686e508c3b774c9ef5bead2eeb5a12f1f84 WatchSource:0}: Error finding container 89962f5dfe9d58bda6e2106e5a5d9686e508c3b774c9ef5bead2eeb5a12f1f84: Status 404 returned error can't find the container with id 89962f5dfe9d58bda6e2106e5a5d9686e508c3b774c9ef5bead2eeb5a12f1f84
Jul 24 04:07:37.978198 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.978155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w" event={"ID":"2abf2f1b-9975-4e5e-9f43-a10c30353462","Type":"ContainerStarted","Data":"e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66"}
Jul 24 04:07:37.978614 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.978210    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w" event={"ID":"2abf2f1b-9975-4e5e-9f43-a10c30353462","Type":"ContainerStarted","Data":"89962f5dfe9d58bda6e2106e5a5d9686e508c3b774c9ef5bead2eeb5a12f1f84"}
Jul 24 04:07:37.999733 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:37.999679    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w" podStartSLOduration=1.99966365 podStartE2EDuration="1.99966365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:07:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:07:37.997558827 +0000 UTC m=+3024.758467308" watchObservedRunningTime="2026-07-24 04:07:37.99966365 +0000 UTC m=+3024.760572109"
Jul 24 04:07:40.985785 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:40.985746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:07:47.257764 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:47.257723    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:47.258350 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:47.257782    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:47.270499 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:47.270471    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:48.041761 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:48.041729    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:07:56.958280 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:07:56.958239    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:08:12.225465 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.225385    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"]
Jul 24 04:08:12.229542 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.229522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.231705 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.231681    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-20e639-serving-cert\""
Jul 24 04:08:12.238594 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.238566    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"]
Jul 24 04:08:12.328085 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.328042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a9dca25-06ab-4db5-84fe-69cb549b79ef-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.328310 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.328099    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a9dca25-06ab-4db5-84fe-69cb549b79ef-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.328310 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.328197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt64m\" (UniqueName: \"kubernetes.io/projected/3a9dca25-06ab-4db5-84fe-69cb549b79ef-kube-api-access-lt64m\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.429468 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.429426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt64m\" (UniqueName: \"kubernetes.io/projected/3a9dca25-06ab-4db5-84fe-69cb549b79ef-kube-api-access-lt64m\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.429468 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.429471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a9dca25-06ab-4db5-84fe-69cb549b79ef-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.429687 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.429514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a9dca25-06ab-4db5-84fe-69cb549b79ef-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.429687 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:08:12.429604    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-serving-cert: secret "maas-api-e2e-ipp-cleanup-20e639-serving-cert" not found
Jul 24 04:08:12.429687 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:08:12.429679    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a9dca25-06ab-4db5-84fe-69cb549b79ef-maas-api-tls podName:3a9dca25-06ab-4db5-84fe-69cb549b79ef nodeName:}" failed. No retries permitted until 2026-07-24 04:08:12.92966083 +0000 UTC m=+3059.690569290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a9dca25-06ab-4db5-84fe-69cb549b79ef-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9" (UID: "3a9dca25-06ab-4db5-84fe-69cb549b79ef") : secret "maas-api-e2e-ipp-cleanup-20e639-serving-cert" not found
Jul 24 04:08:12.430193 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.430173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a9dca25-06ab-4db5-84fe-69cb549b79ef-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.438641 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.438613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt64m\" (UniqueName: \"kubernetes.io/projected/3a9dca25-06ab-4db5-84fe-69cb549b79ef-kube-api-access-lt64m\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.934871 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.934831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a9dca25-06ab-4db5-84fe-69cb549b79ef-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:12.937468 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:12.937440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a9dca25-06ab-4db5-84fe-69cb549b79ef-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:13.142887 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:13.142843    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:13.278286 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:13.278235    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"]
Jul 24 04:08:13.280576 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:08:13.280545    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a9dca25_06ab_4db5_84fe_69cb549b79ef.slice/crio-0eb8340792df924c85739797d55418e997ec3ed8c29e7c4ae29387d45fa84607 WatchSource:0}: Error finding container 0eb8340792df924c85739797d55418e997ec3ed8c29e7c4ae29387d45fa84607: Status 404 returned error can't find the container with id 0eb8340792df924c85739797d55418e997ec3ed8c29e7c4ae29387d45fa84607
Jul 24 04:08:14.170935 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:14.170897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9" event={"ID":"3a9dca25-06ab-4db5-84fe-69cb549b79ef","Type":"ContainerStarted","Data":"fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4"}
Jul 24 04:08:14.171133 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:14.170960    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9" event={"ID":"3a9dca25-06ab-4db5-84fe-69cb549b79ef","Type":"ContainerStarted","Data":"0eb8340792df924c85739797d55418e997ec3ed8c29e7c4ae29387d45fa84607"}
Jul 24 04:08:14.171133 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:14.170986    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:14.190399 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:14.190348    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9" podStartSLOduration=1.816676144 podStartE2EDuration="2.190331956s" totalImagesPullingTime="373.655812ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:08:14.18854037 +0000 UTC m=+3060.949448844" watchObservedRunningTime="2026-07-24 04:08:14.190331956 +0000 UTC m=+3060.951240434"
Jul 24 04:08:18.302876 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:18.302835    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:08:20.181983 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:20.181949    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:27.475566 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:27.475526    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"]
Jul 24 04:08:27.478948 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:27.478930    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"
Jul 24 04:08:27.481233 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:27.481211    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 04:08:27.486317 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:27.485932    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"]
Jul 24 04:08:27.553436 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:27.553400    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de2dbf2-ffe7-40b3-9ca3-bc17941184b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9\" (UID: \"8de2dbf2-ffe7-40b3-9ca3-bc17941184b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"
Jul 24 04:08:27.654646 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:27.654612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de2dbf2-ffe7-40b3-9ca3-bc17941184b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9\" (UID: \"8de2dbf2-ffe7-40b3-9ca3-bc17941184b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"
Jul 24 04:08:27.655181 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:27.655164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de2dbf2-ffe7-40b3-9ca3-bc17941184b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9\" (UID: \"8de2dbf2-ffe7-40b3-9ca3-bc17941184b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"
Jul 24 04:08:27.790696 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:27.790602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"
Jul 24 04:08:27.927422 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:27.927393    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"]
Jul 24 04:08:27.929010 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:08:27.928980    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de2dbf2_ffe7_40b3_9ca3_bc17941184b2.slice/crio-2860dc2feb50ebb8229448eba13b26d6428f03dacfc933a2033724dc00519f5c WatchSource:0}: Error finding container 2860dc2feb50ebb8229448eba13b26d6428f03dacfc933a2033724dc00519f5c: Status 404 returned error can't find the container with id 2860dc2feb50ebb8229448eba13b26d6428f03dacfc933a2033724dc00519f5c
Jul 24 04:08:28.246194 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:28.246156    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9" event={"ID":"8de2dbf2-ffe7-40b3-9ca3-bc17941184b2","Type":"ContainerStarted","Data":"1d7605c4102215ddae9dad982138f30c96c0c4c6e8e3b3eafc9a76aeb30b76d9"}
Jul 24 04:08:28.246388 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:28.246219    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9" event={"ID":"8de2dbf2-ffe7-40b3-9ca3-bc17941184b2","Type":"ContainerStarted","Data":"2860dc2feb50ebb8229448eba13b26d6428f03dacfc933a2033724dc00519f5c"}
Jul 24 04:08:28.261752 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:28.261705    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9" podStartSLOduration=1.261691201 podStartE2EDuration="1.261691201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:08:28.260342385 +0000 UTC m=+3075.021250864" watchObservedRunningTime="2026-07-24 04:08:28.261691201 +0000 UTC m=+3075.022599679"
Jul 24 04:08:29.253800 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:29.253760    2577 generic.go:356] "Generic (PLEG): container finished" podID="8de2dbf2-ffe7-40b3-9ca3-bc17941184b2" containerID="1d7605c4102215ddae9dad982138f30c96c0c4c6e8e3b3eafc9a76aeb30b76d9" exitCode=0
Jul 24 04:08:29.254201 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:29.253848    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9" event={"ID":"8de2dbf2-ffe7-40b3-9ca3-bc17941184b2","Type":"ContainerDied","Data":"1d7605c4102215ddae9dad982138f30c96c0c4c6e8e3b3eafc9a76aeb30b76d9"}
Jul 24 04:08:30.392990 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:30.392964    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"
Jul 24 04:08:30.477691 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:30.477657    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de2dbf2-ffe7-40b3-9ca3-bc17941184b2-maas-api-service-ca\") pod \"8de2dbf2-ffe7-40b3-9ca3-bc17941184b2\" (UID: \"8de2dbf2-ffe7-40b3-9ca3-bc17941184b2\") "
Jul 24 04:08:30.478047 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:30.478021    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8de2dbf2-ffe7-40b3-9ca3-bc17941184b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8de2dbf2-ffe7-40b3-9ca3-bc17941184b2" (UID: "8de2dbf2-ffe7-40b3-9ca3-bc17941184b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:08:30.578784 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:30.578673    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8de2dbf2-ffe7-40b3-9ca3-bc17941184b2-maas-api-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:08:31.264952 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:31.264908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9" event={"ID":"8de2dbf2-ffe7-40b3-9ca3-bc17941184b2","Type":"ContainerDied","Data":"2860dc2feb50ebb8229448eba13b26d6428f03dacfc933a2033724dc00519f5c"}
Jul 24 04:08:31.264952 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:31.264946    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"
Jul 24 04:08:31.265171 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:31.264973    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2860dc2feb50ebb8229448eba13b26d6428f03dacfc933a2033724dc00519f5c"
Jul 24 04:08:38.027396 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.027356    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"]
Jul 24 04:08:38.027822 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.027594    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9" podUID="3a9dca25-06ab-4db5-84fe-69cb549b79ef" containerName="maas-api" containerID="cri-o://fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4" gracePeriod=30
Jul 24 04:08:38.269489 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.269462    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:38.306323 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.306223    2577 generic.go:356] "Generic (PLEG): container finished" podID="3a9dca25-06ab-4db5-84fe-69cb549b79ef" containerID="fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4" exitCode=0
Jul 24 04:08:38.306323 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.306306    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"
Jul 24 04:08:38.306521 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.306301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9" event={"ID":"3a9dca25-06ab-4db5-84fe-69cb549b79ef","Type":"ContainerDied","Data":"fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4"}
Jul 24 04:08:38.306521 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.306430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9" event={"ID":"3a9dca25-06ab-4db5-84fe-69cb549b79ef","Type":"ContainerDied","Data":"0eb8340792df924c85739797d55418e997ec3ed8c29e7c4ae29387d45fa84607"}
Jul 24 04:08:38.306521 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.306453    2577 scope.go:117] "RemoveContainer" containerID="fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4"
Jul 24 04:08:38.315454 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.315436    2577 scope.go:117] "RemoveContainer" containerID="fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4"
Jul 24 04:08:38.315708 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:08:38.315689    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4\": container with ID starting with fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4 not found: ID does not exist" containerID="fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4"
Jul 24 04:08:38.315763 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.315718    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4"} err="failed to get container status \"fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4\": rpc error: code = NotFound desc = could not find container \"fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4\": container with ID starting with fc3b6190f99f2bcaf40024ddacf5b2bbe149f8f3d5cbdd1ad5473504a55dd4e4 not found: ID does not exist"
Jul 24 04:08:38.346390 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.346354    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a9dca25-06ab-4db5-84fe-69cb549b79ef-openshift-service-ca\") pod \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") "
Jul 24 04:08:38.346390 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.346395    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt64m\" (UniqueName: \"kubernetes.io/projected/3a9dca25-06ab-4db5-84fe-69cb549b79ef-kube-api-access-lt64m\") pod \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") "
Jul 24 04:08:38.346608 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.346421    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a9dca25-06ab-4db5-84fe-69cb549b79ef-maas-api-tls\") pod \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\" (UID: \"3a9dca25-06ab-4db5-84fe-69cb549b79ef\") "
Jul 24 04:08:38.346735 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.346713    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a9dca25-06ab-4db5-84fe-69cb549b79ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3a9dca25-06ab-4db5-84fe-69cb549b79ef" (UID: "3a9dca25-06ab-4db5-84fe-69cb549b79ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:08:38.348626 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.348603    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a9dca25-06ab-4db5-84fe-69cb549b79ef-kube-api-access-lt64m" (OuterVolumeSpecName: "kube-api-access-lt64m") pod "3a9dca25-06ab-4db5-84fe-69cb549b79ef" (UID: "3a9dca25-06ab-4db5-84fe-69cb549b79ef"). InnerVolumeSpecName "kube-api-access-lt64m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:08:38.348730 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.348626    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a9dca25-06ab-4db5-84fe-69cb549b79ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3a9dca25-06ab-4db5-84fe-69cb549b79ef" (UID: "3a9dca25-06ab-4db5-84fe-69cb549b79ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:08:38.447457 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.447415    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a9dca25-06ab-4db5-84fe-69cb549b79ef-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:08:38.447457 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.447449    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt64m\" (UniqueName: \"kubernetes.io/projected/3a9dca25-06ab-4db5-84fe-69cb549b79ef-kube-api-access-lt64m\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:08:38.447457 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.447462    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a9dca25-06ab-4db5-84fe-69cb549b79ef-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:08:38.627921 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.627884    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"]
Jul 24 04:08:38.633534 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:38.633509    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-20e639-c58c8fdc4-lvfz9"]
Jul 24 04:08:39.642229 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:39.642186    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a9dca25-06ab-4db5-84fe-69cb549b79ef" path="/var/lib/kubelet/pods/3a9dca25-06ab-4db5-84fe-69cb549b79ef/volumes"
Jul 24 04:08:42.551988 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:42.551950    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"]
Jul 24 04:08:42.554445 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:42.554420    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-20e639-5xjw9"]
Jul 24 04:08:42.795512 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:42.795481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:08:43.642502 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:43.642471    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8de2dbf2-ffe7-40b3-9ca3-bc17941184b2" path="/var/lib/kubelet/pods/8de2dbf2-ffe7-40b3-9ca3-bc17941184b2/volumes"
Jul 24 04:08:52.735851 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.735812    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"]
Jul 24 04:08:52.736372 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.736149    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de2dbf2-ffe7-40b3-9ca3-bc17941184b2" containerName="revoke-keys"
Jul 24 04:08:52.736372 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.736163    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de2dbf2-ffe7-40b3-9ca3-bc17941184b2" containerName="revoke-keys"
Jul 24 04:08:52.736372 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.736180    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a9dca25-06ab-4db5-84fe-69cb549b79ef" containerName="maas-api"
Jul 24 04:08:52.736372 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.736186    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a9dca25-06ab-4db5-84fe-69cb549b79ef" containerName="maas-api"
Jul 24 04:08:52.736372 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.736278    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8de2dbf2-ffe7-40b3-9ca3-bc17941184b2" containerName="revoke-keys"
Jul 24 04:08:52.736372 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.736286    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a9dca25-06ab-4db5-84fe-69cb549b79ef" containerName="maas-api"
Jul 24 04:08:52.740746 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.740722    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"
Jul 24 04:08:52.743369 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.743335    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 04:08:52.745971 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.745939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"]
Jul 24 04:08:52.876794 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.876753    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e9ec19-9fe0-47c8-a16c-e54a1038cc50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq\" (UID: \"f8e9ec19-9fe0-47c8-a16c-e54a1038cc50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"
Jul 24 04:08:52.977743 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.977701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e9ec19-9fe0-47c8-a16c-e54a1038cc50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq\" (UID: \"f8e9ec19-9fe0-47c8-a16c-e54a1038cc50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"
Jul 24 04:08:52.978426 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:52.978403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e9ec19-9fe0-47c8-a16c-e54a1038cc50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq\" (UID: \"f8e9ec19-9fe0-47c8-a16c-e54a1038cc50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"
Jul 24 04:08:53.054551 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:53.054464    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"
Jul 24 04:08:53.193980 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:53.193952    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"]
Jul 24 04:08:53.195979 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:08:53.195951    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e9ec19_9fe0_47c8_a16c_e54a1038cc50.slice/crio-8b118771aa62217e0272ce72df326514f78bf29f7cec39d5bd9db22b8665da21 WatchSource:0}: Error finding container 8b118771aa62217e0272ce72df326514f78bf29f7cec39d5bd9db22b8665da21: Status 404 returned error can't find the container with id 8b118771aa62217e0272ce72df326514f78bf29f7cec39d5bd9db22b8665da21
Jul 24 04:08:53.387751 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:53.387647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq" event={"ID":"f8e9ec19-9fe0-47c8-a16c-e54a1038cc50","Type":"ContainerStarted","Data":"8b118771aa62217e0272ce72df326514f78bf29f7cec39d5bd9db22b8665da21"}
Jul 24 04:08:54.394575 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:54.394539    2577 generic.go:356] "Generic (PLEG): container finished" podID="f8e9ec19-9fe0-47c8-a16c-e54a1038cc50" containerID="8a4ea2871cdfb3b8f6c51b293b70328943ce5acf0065e9c3f662b67840199ac3" exitCode=0
Jul 24 04:08:54.395026 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:54.394630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq" event={"ID":"f8e9ec19-9fe0-47c8-a16c-e54a1038cc50","Type":"ContainerDied","Data":"8a4ea2871cdfb3b8f6c51b293b70328943ce5acf0065e9c3f662b67840199ac3"}
Jul 24 04:08:55.536420 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:55.536394    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"
Jul 24 04:08:55.598553 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:55.598511    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e9ec19-9fe0-47c8-a16c-e54a1038cc50-maas-api-service-ca\") pod \"f8e9ec19-9fe0-47c8-a16c-e54a1038cc50\" (UID: \"f8e9ec19-9fe0-47c8-a16c-e54a1038cc50\") "
Jul 24 04:08:55.598909 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:55.598890    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8e9ec19-9fe0-47c8-a16c-e54a1038cc50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8e9ec19-9fe0-47c8-a16c-e54a1038cc50" (UID: "f8e9ec19-9fe0-47c8-a16c-e54a1038cc50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:08:55.699451 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:55.699408    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e9ec19-9fe0-47c8-a16c-e54a1038cc50-maas-api-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:08:56.407410 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:56.407382    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"
Jul 24 04:08:56.407593 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:56.407383    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq" event={"ID":"f8e9ec19-9fe0-47c8-a16c-e54a1038cc50","Type":"ContainerDied","Data":"8b118771aa62217e0272ce72df326514f78bf29f7cec39d5bd9db22b8665da21"}
Jul 24 04:08:56.407593 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:08:56.407510    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b118771aa62217e0272ce72df326514f78bf29f7cec39d5bd9db22b8665da21"
Jul 24 04:09:07.487701 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:07.487662    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:09:08.274665 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.274627    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"]
Jul 24 04:09:08.274943 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.274888    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x" podUID="dbc3d6fb-5fc4-4cc7-9f70-ca288750641e" containerName="maas-api" containerID="cri-o://f2561553b32c802f79e7a83ffcefda9c8eb472c8a0b48cf43f91ef088503aef4" gracePeriod=30
Jul 24 04:09:08.471725 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.471692    2577 generic.go:356] "Generic (PLEG): container finished" podID="dbc3d6fb-5fc4-4cc7-9f70-ca288750641e" containerID="f2561553b32c802f79e7a83ffcefda9c8eb472c8a0b48cf43f91ef088503aef4" exitCode=0
Jul 24 04:09:08.471903 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.471761    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x" event={"ID":"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e","Type":"ContainerDied","Data":"f2561553b32c802f79e7a83ffcefda9c8eb472c8a0b48cf43f91ef088503aef4"}
Jul 24 04:09:08.532566 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.532498    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:09:08.614394 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.614355    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-openshift-service-ca\") pod \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") "
Jul 24 04:09:08.614600 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.614450    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-maas-api-tls\") pod \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") "
Jul 24 04:09:08.614600 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.614514    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vkw6\" (UniqueName: \"kubernetes.io/projected/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-kube-api-access-8vkw6\") pod \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\" (UID: \"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e\") "
Jul 24 04:09:08.614819 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.614787    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dbc3d6fb-5fc4-4cc7-9f70-ca288750641e" (UID: "dbc3d6fb-5fc4-4cc7-9f70-ca288750641e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:09:08.616840 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.616806    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-kube-api-access-8vkw6" (OuterVolumeSpecName: "kube-api-access-8vkw6") pod "dbc3d6fb-5fc4-4cc7-9f70-ca288750641e" (UID: "dbc3d6fb-5fc4-4cc7-9f70-ca288750641e"). InnerVolumeSpecName "kube-api-access-8vkw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:09:08.616939 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.616850    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dbc3d6fb-5fc4-4cc7-9f70-ca288750641e" (UID: "dbc3d6fb-5fc4-4cc7-9f70-ca288750641e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:09:08.715657 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.715595    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:09:08.715657 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.715647    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vkw6\" (UniqueName: \"kubernetes.io/projected/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-kube-api-access-8vkw6\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:09:08.715657 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:08.715662    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:09:09.478718 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:09.478682    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x" event={"ID":"dbc3d6fb-5fc4-4cc7-9f70-ca288750641e","Type":"ContainerDied","Data":"0a3bea5be9e556b295ebf66522b77de8ce76807ad85789598bef07d69bef317e"}
Jul 24 04:09:09.478900 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:09.478749    2577 scope.go:117] "RemoveContainer" containerID="f2561553b32c802f79e7a83ffcefda9c8eb472c8a0b48cf43f91ef088503aef4"
Jul 24 04:09:09.478900 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:09.478694    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"
Jul 24 04:09:09.499698 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:09.499661    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"]
Jul 24 04:09:09.503628 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:09.503603    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-733dde-bd7df44f6-fn78x"]
Jul 24 04:09:09.643044 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:09.643014    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbc3d6fb-5fc4-4cc7-9f70-ca288750641e" path="/var/lib/kubelet/pods/dbc3d6fb-5fc4-4cc7-9f70-ca288750641e/volumes"
Jul 24 04:09:12.818939 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:12.818901    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"]
Jul 24 04:09:12.821638 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:12.821611    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-733dde-gsrjq"]
Jul 24 04:09:13.644035 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:13.644000    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8e9ec19-9fe0-47c8-a16c-e54a1038cc50" path="/var/lib/kubelet/pods/f8e9ec19-9fe0-47c8-a16c-e54a1038cc50/volumes"
Jul 24 04:09:16.990550 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:16.990507    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:09:18.190567 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:18.190531    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"]
Jul 24 04:09:33.210746 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.210686    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w" podUID="2abf2f1b-9975-4e5e-9f43-a10c30353462" containerName="main" containerID="cri-o://e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66" gracePeriod=45
Jul 24 04:09:33.472256 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.472173    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:09:33.526442 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.526403    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2abf2f1b-9975-4e5e-9f43-a10c30353462-tls-certs\") pod \"2abf2f1b-9975-4e5e-9f43-a10c30353462\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") "
Jul 24 04:09:33.526639 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.526499    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-dshm\") pod \"2abf2f1b-9975-4e5e-9f43-a10c30353462\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") "
Jul 24 04:09:33.526639 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.526561    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7d5rc\" (UniqueName: \"kubernetes.io/projected/2abf2f1b-9975-4e5e-9f43-a10c30353462-kube-api-access-7d5rc\") pod \"2abf2f1b-9975-4e5e-9f43-a10c30353462\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") "
Jul 24 04:09:33.526639 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.526602    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-tmp-dir\") pod \"2abf2f1b-9975-4e5e-9f43-a10c30353462\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") "
Jul 24 04:09:33.526824 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.526652    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-home\") pod \"2abf2f1b-9975-4e5e-9f43-a10c30353462\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") "
Jul 24 04:09:33.526824 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.526689    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-model-cache\") pod \"2abf2f1b-9975-4e5e-9f43-a10c30353462\" (UID: \"2abf2f1b-9975-4e5e-9f43-a10c30353462\") "
Jul 24 04:09:33.526962 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.526927    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-home" (OuterVolumeSpecName: "home") pod "2abf2f1b-9975-4e5e-9f43-a10c30353462" (UID: "2abf2f1b-9975-4e5e-9f43-a10c30353462"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:09:33.526962 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.526940    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2abf2f1b-9975-4e5e-9f43-a10c30353462" (UID: "2abf2f1b-9975-4e5e-9f43-a10c30353462"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:09:33.527196 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.527133    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-model-cache" (OuterVolumeSpecName: "model-cache") pod "2abf2f1b-9975-4e5e-9f43-a10c30353462" (UID: "2abf2f1b-9975-4e5e-9f43-a10c30353462"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:09:33.528964 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.528934    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2abf2f1b-9975-4e5e-9f43-a10c30353462-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2abf2f1b-9975-4e5e-9f43-a10c30353462" (UID: "2abf2f1b-9975-4e5e-9f43-a10c30353462"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:09:33.529074 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.528976    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2abf2f1b-9975-4e5e-9f43-a10c30353462-kube-api-access-7d5rc" (OuterVolumeSpecName: "kube-api-access-7d5rc") pod "2abf2f1b-9975-4e5e-9f43-a10c30353462" (UID: "2abf2f1b-9975-4e5e-9f43-a10c30353462"). InnerVolumeSpecName "kube-api-access-7d5rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:09:33.529133 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.529120    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-dshm" (OuterVolumeSpecName: "dshm") pod "2abf2f1b-9975-4e5e-9f43-a10c30353462" (UID: "2abf2f1b-9975-4e5e-9f43-a10c30353462"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 04:09:33.606753 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.606717    2577 generic.go:356] "Generic (PLEG): container finished" podID="2abf2f1b-9975-4e5e-9f43-a10c30353462" containerID="e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66" exitCode=0
Jul 24 04:09:33.606753 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.606759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w" event={"ID":"2abf2f1b-9975-4e5e-9f43-a10c30353462","Type":"ContainerDied","Data":"e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66"}
Jul 24 04:09:33.607030 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.606795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w" event={"ID":"2abf2f1b-9975-4e5e-9f43-a10c30353462","Type":"ContainerDied","Data":"89962f5dfe9d58bda6e2106e5a5d9686e508c3b774c9ef5bead2eeb5a12f1f84"}
Jul 24 04:09:33.607030 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.606791    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"
Jul 24 04:09:33.607030 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.606809    2577 scope.go:117] "RemoveContainer" containerID="e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66"
Jul 24 04:09:33.616521 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.616499    2577 scope.go:117] "RemoveContainer" containerID="e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66"
Jul 24 04:09:33.616819 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:09:33.616798    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66\": container with ID starting with e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66 not found: ID does not exist" containerID="e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66"
Jul 24 04:09:33.616897 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.616841    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66"} err="failed to get container status \"e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66\": rpc error: code = NotFound desc = could not find container \"e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66\": container with ID starting with e47be8425d33b48f982fa0dffe29024afdf78d1961e1c36ccad8eec37673bf66 not found: ID does not exist"
Jul 24 04:09:33.627524 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.627490    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2abf2f1b-9975-4e5e-9f43-a10c30353462-tls-certs\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:09:33.627524 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.627523    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-dshm\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:09:33.627736 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.627536    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d5rc\" (UniqueName: \"kubernetes.io/projected/2abf2f1b-9975-4e5e-9f43-a10c30353462-kube-api-access-7d5rc\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:09:33.627736 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.627548    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-tmp-dir\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:09:33.627736 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.627559    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-home\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:09:33.627736 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.627569    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2abf2f1b-9975-4e5e-9f43-a10c30353462-model-cache\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:09:33.628687 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.628663    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"]
Jul 24 04:09:33.632436 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.632410    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-733dde/ipp-route-733dde-kserve-5f4d48fbbc-5k49w"]
Jul 24 04:09:33.642322 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:09:33.642288    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2abf2f1b-9975-4e5e-9f43-a10c30353462" path="/var/lib/kubelet/pods/2abf2f1b-9975-4e5e-9f43-a10c30353462/volumes"
Jul 24 04:10:01.666187 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666140    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"]
Jul 24 04:10:01.666703 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666642    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbc3d6fb-5fc4-4cc7-9f70-ca288750641e" containerName="maas-api"
Jul 24 04:10:01.666703 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666661    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbc3d6fb-5fc4-4cc7-9f70-ca288750641e" containerName="maas-api"
Jul 24 04:10:01.666703 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666674    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2abf2f1b-9975-4e5e-9f43-a10c30353462" containerName="main"
Jul 24 04:10:01.666703 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666682    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2abf2f1b-9975-4e5e-9f43-a10c30353462" containerName="main"
Jul 24 04:10:01.666703 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666700    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8e9ec19-9fe0-47c8-a16c-e54a1038cc50" containerName="revoke-keys"
Jul 24 04:10:01.666961 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666710    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e9ec19-9fe0-47c8-a16c-e54a1038cc50" containerName="revoke-keys"
Jul 24 04:10:01.666961 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666812    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbc3d6fb-5fc4-4cc7-9f70-ca288750641e" containerName="maas-api"
Jul 24 04:10:01.666961 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666824    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8e9ec19-9fe0-47c8-a16c-e54a1038cc50" containerName="revoke-keys"
Jul 24 04:10:01.666961 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.666836    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2abf2f1b-9975-4e5e-9f43-a10c30353462" containerName="main"
Jul 24 04:10:01.669895 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.669874    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"
Jul 24 04:10:01.672333 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.672313    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 04:10:01.676627 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.676558    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"]
Jul 24 04:10:01.781718 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.781680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df112e2d-a85e-4d01-8734-9868d03e12da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm\" (UID: \"df112e2d-a85e-4d01-8734-9868d03e12da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"
Jul 24 04:10:01.883124 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.883066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df112e2d-a85e-4d01-8734-9868d03e12da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm\" (UID: \"df112e2d-a85e-4d01-8734-9868d03e12da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"
Jul 24 04:10:01.883785 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.883765    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df112e2d-a85e-4d01-8734-9868d03e12da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm\" (UID: \"df112e2d-a85e-4d01-8734-9868d03e12da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"
Jul 24 04:10:01.981234 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:01.981140    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"
Jul 24 04:10:02.116024 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:02.115989    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"]
Jul 24 04:10:02.118415 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:10:02.118387    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf112e2d_a85e_4d01_8734_9868d03e12da.slice/crio-b0cb6b0e07276bfa9f70861318cf565904d8f581481cce3846382667f9fb7484 WatchSource:0}: Error finding container b0cb6b0e07276bfa9f70861318cf565904d8f581481cce3846382667f9fb7484: Status 404 returned error can't find the container with id b0cb6b0e07276bfa9f70861318cf565904d8f581481cce3846382667f9fb7484
Jul 24 04:10:02.763318 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:02.763258    2577 generic.go:356] "Generic (PLEG): container finished" podID="df112e2d-a85e-4d01-8734-9868d03e12da" containerID="11130ae5bbe802d852cc24b5188aa9864fc3502ba7edb9155bf0d7a3b6899d74" exitCode=0
Jul 24 04:10:02.763730 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:02.763332    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm" event={"ID":"df112e2d-a85e-4d01-8734-9868d03e12da","Type":"ContainerDied","Data":"11130ae5bbe802d852cc24b5188aa9864fc3502ba7edb9155bf0d7a3b6899d74"}
Jul 24 04:10:02.763730 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:02.763387    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm" event={"ID":"df112e2d-a85e-4d01-8734-9868d03e12da","Type":"ContainerStarted","Data":"b0cb6b0e07276bfa9f70861318cf565904d8f581481cce3846382667f9fb7484"}
Jul 24 04:10:03.899413 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:03.899382    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"
Jul 24 04:10:03.903315 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:03.903277    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df112e2d-a85e-4d01-8734-9868d03e12da-maas-api-service-ca\") pod \"df112e2d-a85e-4d01-8734-9868d03e12da\" (UID: \"df112e2d-a85e-4d01-8734-9868d03e12da\") "
Jul 24 04:10:03.903568 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:03.903547    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df112e2d-a85e-4d01-8734-9868d03e12da-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df112e2d-a85e-4d01-8734-9868d03e12da" (UID: "df112e2d-a85e-4d01-8734-9868d03e12da"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:04.003829 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:04.003791    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df112e2d-a85e-4d01-8734-9868d03e12da-maas-api-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:04.774914 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:04.774889    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"
Jul 24 04:10:04.775155 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:04.774879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm" event={"ID":"df112e2d-a85e-4d01-8734-9868d03e12da","Type":"ContainerDied","Data":"b0cb6b0e07276bfa9f70861318cf565904d8f581481cce3846382667f9fb7484"}
Jul 24 04:10:04.775155 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:04.775009    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0cb6b0e07276bfa9f70861318cf565904d8f581481cce3846382667f9fb7484"
Jul 24 04:10:12.167845 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.167809    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"]
Jul 24 04:10:12.168280 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.168050    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw" podUID="25d14fd0-b7b4-484c-a1f0-3669f872d8f0" containerName="maas-api" containerID="cri-o://3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b" gracePeriod=30
Jul 24 04:10:12.411354 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.411329    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:10:12.470053 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.469940    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgznv\" (UniqueName: \"kubernetes.io/projected/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-kube-api-access-cgznv\") pod \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") "
Jul 24 04:10:12.470053 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.470016    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-maas-api-tls\") pod \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") "
Jul 24 04:10:12.470053 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.470039    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-openshift-service-ca\") pod \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\" (UID: \"25d14fd0-b7b4-484c-a1f0-3669f872d8f0\") "
Jul 24 04:10:12.470442 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.470421    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "25d14fd0-b7b4-484c-a1f0-3669f872d8f0" (UID: "25d14fd0-b7b4-484c-a1f0-3669f872d8f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:12.472347 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.472322    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-kube-api-access-cgznv" (OuterVolumeSpecName: "kube-api-access-cgznv") pod "25d14fd0-b7b4-484c-a1f0-3669f872d8f0" (UID: "25d14fd0-b7b4-484c-a1f0-3669f872d8f0"). InnerVolumeSpecName "kube-api-access-cgznv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:10:12.472432 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.472408    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "25d14fd0-b7b4-484c-a1f0-3669f872d8f0" (UID: "25d14fd0-b7b4-484c-a1f0-3669f872d8f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:10:12.571017 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.570972    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:12.571017 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.571015    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgznv\" (UniqueName: \"kubernetes.io/projected/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-kube-api-access-cgznv\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:12.571017 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.571026    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d14fd0-b7b4-484c-a1f0-3669f872d8f0-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:12.816384 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.816284    2577 generic.go:356] "Generic (PLEG): container finished" podID="25d14fd0-b7b4-484c-a1f0-3669f872d8f0" containerID="3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b" exitCode=0
Jul 24 04:10:12.816384 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.816329    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw" event={"ID":"25d14fd0-b7b4-484c-a1f0-3669f872d8f0","Type":"ContainerDied","Data":"3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b"}
Jul 24 04:10:12.816628 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.816402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw" event={"ID":"25d14fd0-b7b4-484c-a1f0-3669f872d8f0","Type":"ContainerDied","Data":"3ab9c21aad445b7f667e5d3e3fc6c1e898a90a0c1e3fe441e1ab098b17461a4a"}
Jul 24 04:10:12.816628 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.816342    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"
Jul 24 04:10:12.816628 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.816434    2577 scope.go:117] "RemoveContainer" containerID="3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b"
Jul 24 04:10:12.826119 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.826096    2577 scope.go:117] "RemoveContainer" containerID="3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b"
Jul 24 04:10:12.826403 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:10:12.826384    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b\": container with ID starting with 3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b not found: ID does not exist" containerID="3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b"
Jul 24 04:10:12.826476 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.826410    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b"} err="failed to get container status \"3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b\": rpc error: code = NotFound desc = could not find container \"3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b\": container with ID starting with 3d06414ed0cc758965592be27727edbdeab7864daccf6d65b76cdf3a2c2be71b not found: ID does not exist"
Jul 24 04:10:12.837577 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.837552    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"]
Jul 24 04:10:12.841121 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:12.841097    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0cae5e-85d47dfcb4-qjqzw"]
Jul 24 04:10:13.642929 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:13.642889    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25d14fd0-b7b4-484c-a1f0-3669f872d8f0" path="/var/lib/kubelet/pods/25d14fd0-b7b4-484c-a1f0-3669f872d8f0/volumes"
Jul 24 04:10:15.687847 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:15.687811    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:10:16.747442 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:16.747403    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"]
Jul 24 04:10:16.753482 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:16.753451    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0cae5e-mxdqm"]
Jul 24 04:10:17.642553 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:17.642519    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df112e2d-a85e-4d01-8734-9868d03e12da" path="/var/lib/kubelet/pods/df112e2d-a85e-4d01-8734-9868d03e12da/volumes"
Jul 24 04:10:23.499723 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.499682    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"]
Jul 24 04:10:23.500121 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.500031    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df112e2d-a85e-4d01-8734-9868d03e12da" containerName="revoke-keys"
Jul 24 04:10:23.500121 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.500043    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="df112e2d-a85e-4d01-8734-9868d03e12da" containerName="revoke-keys"
Jul 24 04:10:23.500121 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.500055    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25d14fd0-b7b4-484c-a1f0-3669f872d8f0" containerName="maas-api"
Jul 24 04:10:23.500121 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.500061    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="25d14fd0-b7b4-484c-a1f0-3669f872d8f0" containerName="maas-api"
Jul 24 04:10:23.500121 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.500119    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="25d14fd0-b7b4-484c-a1f0-3669f872d8f0" containerName="maas-api"
Jul 24 04:10:23.500330 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.500130    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="df112e2d-a85e-4d01-8734-9868d03e12da" containerName="revoke-keys"
Jul 24 04:10:23.504469 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.504449    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"
Jul 24 04:10:23.507000 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.506970    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 04:10:23.510104 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.510081    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"]
Jul 24 04:10:23.563203 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.563167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390156e5-e2e3-4fa6-91a0-6af686577e44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk\" (UID: \"390156e5-e2e3-4fa6-91a0-6af686577e44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"
Jul 24 04:10:23.663659 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.663610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390156e5-e2e3-4fa6-91a0-6af686577e44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk\" (UID: \"390156e5-e2e3-4fa6-91a0-6af686577e44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"
Jul 24 04:10:23.664243 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.664223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390156e5-e2e3-4fa6-91a0-6af686577e44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk\" (UID: \"390156e5-e2e3-4fa6-91a0-6af686577e44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"
Jul 24 04:10:23.816865 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.816763    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"
Jul 24 04:10:23.954146 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:23.954116    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"]
Jul 24 04:10:23.956110 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:10:23.956079    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390156e5_e2e3_4fa6_91a0_6af686577e44.slice/crio-d233c9465ead70b3345e9cefec910b19ac05a317baacc6eaacbef930e0b1520e WatchSource:0}: Error finding container d233c9465ead70b3345e9cefec910b19ac05a317baacc6eaacbef930e0b1520e: Status 404 returned error can't find the container with id d233c9465ead70b3345e9cefec910b19ac05a317baacc6eaacbef930e0b1520e
Jul 24 04:10:24.883949 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:24.883913    2577 generic.go:356] "Generic (PLEG): container finished" podID="390156e5-e2e3-4fa6-91a0-6af686577e44" containerID="65d2081150c22409ea1dbdb842b10ab23b8199f475124754b2e3cceb0855cb90" exitCode=0
Jul 24 04:10:24.884357 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:24.884003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk" event={"ID":"390156e5-e2e3-4fa6-91a0-6af686577e44","Type":"ContainerDied","Data":"65d2081150c22409ea1dbdb842b10ab23b8199f475124754b2e3cceb0855cb90"}
Jul 24 04:10:24.884357 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:24.884052    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk" event={"ID":"390156e5-e2e3-4fa6-91a0-6af686577e44","Type":"ContainerStarted","Data":"d233c9465ead70b3345e9cefec910b19ac05a317baacc6eaacbef930e0b1520e"}
Jul 24 04:10:26.024028 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:26.023999    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"
Jul 24 04:10:26.082797 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:26.082754    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390156e5-e2e3-4fa6-91a0-6af686577e44-maas-api-service-ca\") pod \"390156e5-e2e3-4fa6-91a0-6af686577e44\" (UID: \"390156e5-e2e3-4fa6-91a0-6af686577e44\") "
Jul 24 04:10:26.083143 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:26.083120    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390156e5-e2e3-4fa6-91a0-6af686577e44-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "390156e5-e2e3-4fa6-91a0-6af686577e44" (UID: "390156e5-e2e3-4fa6-91a0-6af686577e44"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:26.183931 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:26.183892    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/390156e5-e2e3-4fa6-91a0-6af686577e44-maas-api-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:26.898022 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:26.897984    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk" event={"ID":"390156e5-e2e3-4fa6-91a0-6af686577e44","Type":"ContainerDied","Data":"d233c9465ead70b3345e9cefec910b19ac05a317baacc6eaacbef930e0b1520e"}
Jul 24 04:10:26.898022 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:26.898024    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"
Jul 24 04:10:26.898209 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:26.898038    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d233c9465ead70b3345e9cefec910b19ac05a317baacc6eaacbef930e0b1520e"
Jul 24 04:10:33.904767 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:33.904729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"]
Jul 24 04:10:33.905334 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:33.904967    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf" podUID="cdfcdf50-4c24-44cc-b550-7e10acede653" containerName="maas-api" containerID="cri-o://da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2" gracePeriod=30
Jul 24 04:10:34.180162 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.180132    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:10:34.252538 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.252499    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6fvn\" (UniqueName: \"kubernetes.io/projected/cdfcdf50-4c24-44cc-b550-7e10acede653-kube-api-access-x6fvn\") pod \"cdfcdf50-4c24-44cc-b550-7e10acede653\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") "
Jul 24 04:10:34.252730 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.252563    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdfcdf50-4c24-44cc-b550-7e10acede653-openshift-service-ca\") pod \"cdfcdf50-4c24-44cc-b550-7e10acede653\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") "
Jul 24 04:10:34.252730 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.252636    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdfcdf50-4c24-44cc-b550-7e10acede653-maas-api-tls\") pod \"cdfcdf50-4c24-44cc-b550-7e10acede653\" (UID: \"cdfcdf50-4c24-44cc-b550-7e10acede653\") "
Jul 24 04:10:34.252989 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.252956    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdfcdf50-4c24-44cc-b550-7e10acede653-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cdfcdf50-4c24-44cc-b550-7e10acede653" (UID: "cdfcdf50-4c24-44cc-b550-7e10acede653"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:34.254867 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.254841    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdfcdf50-4c24-44cc-b550-7e10acede653-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cdfcdf50-4c24-44cc-b550-7e10acede653" (UID: "cdfcdf50-4c24-44cc-b550-7e10acede653"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:10:34.254969 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.254863    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdfcdf50-4c24-44cc-b550-7e10acede653-kube-api-access-x6fvn" (OuterVolumeSpecName: "kube-api-access-x6fvn") pod "cdfcdf50-4c24-44cc-b550-7e10acede653" (UID: "cdfcdf50-4c24-44cc-b550-7e10acede653"). InnerVolumeSpecName "kube-api-access-x6fvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:10:34.353676 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.353631    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdfcdf50-4c24-44cc-b550-7e10acede653-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:34.353676 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.353668    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdfcdf50-4c24-44cc-b550-7e10acede653-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:34.353676 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.353678    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6fvn\" (UniqueName: \"kubernetes.io/projected/cdfcdf50-4c24-44cc-b550-7e10acede653-kube-api-access-x6fvn\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:34.939723 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.939691    2577 generic.go:356] "Generic (PLEG): container finished" podID="cdfcdf50-4c24-44cc-b550-7e10acede653" containerID="da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2" exitCode=0
Jul 24 04:10:34.940147 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.939735    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"
Jul 24 04:10:34.940147 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.939771    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf" event={"ID":"cdfcdf50-4c24-44cc-b550-7e10acede653","Type":"ContainerDied","Data":"da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2"}
Jul 24 04:10:34.940147 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.939821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf" event={"ID":"cdfcdf50-4c24-44cc-b550-7e10acede653","Type":"ContainerDied","Data":"70e3b204cabdf48456f20e4bbbec36fa87dac8d463c787cc40b3a44dd5a136ec"}
Jul 24 04:10:34.940147 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.939843    2577 scope.go:117] "RemoveContainer" containerID="da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2"
Jul 24 04:10:34.949585 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.949560    2577 scope.go:117] "RemoveContainer" containerID="da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2"
Jul 24 04:10:34.949894 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:10:34.949874    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2\": container with ID starting with da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2 not found: ID does not exist" containerID="da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2"
Jul 24 04:10:34.949951 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.949904    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2"} err="failed to get container status \"da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2\": rpc error: code = NotFound desc = could not find container \"da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2\": container with ID starting with da9747634b5b4ea61c28c39090f62702ebfdd8827cf09eb5aa075851e2bddad2 not found: ID does not exist"
Jul 24 04:10:34.968049 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.967974    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"]
Jul 24 04:10:34.969999 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:34.969972    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5b56b9-5b68c578df-rpbtf"]
Jul 24 04:10:35.642402 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:35.642366    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdfcdf50-4c24-44cc-b550-7e10acede653" path="/var/lib/kubelet/pods/cdfcdf50-4c24-44cc-b550-7e10acede653/volumes"
Jul 24 04:10:37.195179 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:37.195140    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:10:38.622317 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:38.622284    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"]
Jul 24 04:10:38.625206 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:38.625176    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5b56b9-nklbk"]
Jul 24 04:10:39.642594 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:39.642560    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="390156e5-e2e3-4fa6-91a0-6af686577e44" path="/var/lib/kubelet/pods/390156e5-e2e3-4fa6-91a0-6af686577e44/volumes"
Jul 24 04:10:45.882599 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.882552    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"]
Jul 24 04:10:45.883095 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.883024    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="390156e5-e2e3-4fa6-91a0-6af686577e44" containerName="revoke-keys"
Jul 24 04:10:45.883095 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.883043    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="390156e5-e2e3-4fa6-91a0-6af686577e44" containerName="revoke-keys"
Jul 24 04:10:45.883095 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.883067    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdfcdf50-4c24-44cc-b550-7e10acede653" containerName="maas-api"
Jul 24 04:10:45.883095 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.883075    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdfcdf50-4c24-44cc-b550-7e10acede653" containerName="maas-api"
Jul 24 04:10:45.883358 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.883153    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="390156e5-e2e3-4fa6-91a0-6af686577e44" containerName="revoke-keys"
Jul 24 04:10:45.883358 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.883168    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdfcdf50-4c24-44cc-b550-7e10acede653" containerName="maas-api"
Jul 24 04:10:45.887639 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.887618    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"
Jul 24 04:10:45.890093 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.890068    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 04:10:45.893908 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.893879    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"]
Jul 24 04:10:45.960631 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:45.960589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fade9390-0055-4c73-aaab-e32a1b8e4fd3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q\" (UID: \"fade9390-0055-4c73-aaab-e32a1b8e4fd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"
Jul 24 04:10:46.061249 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:46.061192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fade9390-0055-4c73-aaab-e32a1b8e4fd3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q\" (UID: \"fade9390-0055-4c73-aaab-e32a1b8e4fd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"
Jul 24 04:10:46.061874 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:46.061851    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fade9390-0055-4c73-aaab-e32a1b8e4fd3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q\" (UID: \"fade9390-0055-4c73-aaab-e32a1b8e4fd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"
Jul 24 04:10:46.199864 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:46.199829    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"
Jul 24 04:10:46.338533 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:46.338498    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"]
Jul 24 04:10:46.339352 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:10:46.339327    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfade9390_0055_4c73_aaab_e32a1b8e4fd3.slice/crio-8a2af43f49133f31cdc9e2f5f20c64d56b4ad20ab75ee2efe6ca6f454aafad75 WatchSource:0}: Error finding container 8a2af43f49133f31cdc9e2f5f20c64d56b4ad20ab75ee2efe6ca6f454aafad75: Status 404 returned error can't find the container with id 8a2af43f49133f31cdc9e2f5f20c64d56b4ad20ab75ee2efe6ca6f454aafad75
Jul 24 04:10:47.003576 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:47.003544    2577 generic.go:356] "Generic (PLEG): container finished" podID="fade9390-0055-4c73-aaab-e32a1b8e4fd3" containerID="42738811ae1fc29d6097f746fafcf07b6ad9de1d719daf8b6ae6b1e65fd27e97" exitCode=0
Jul 24 04:10:47.003962 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:47.003585    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q" event={"ID":"fade9390-0055-4c73-aaab-e32a1b8e4fd3","Type":"ContainerDied","Data":"42738811ae1fc29d6097f746fafcf07b6ad9de1d719daf8b6ae6b1e65fd27e97"}
Jul 24 04:10:47.003962 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:47.003630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q" event={"ID":"fade9390-0055-4c73-aaab-e32a1b8e4fd3","Type":"ContainerStarted","Data":"8a2af43f49133f31cdc9e2f5f20c64d56b4ad20ab75ee2efe6ca6f454aafad75"}
Jul 24 04:10:48.141509 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:48.141477    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"
Jul 24 04:10:48.282909 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:48.282815    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fade9390-0055-4c73-aaab-e32a1b8e4fd3-maas-api-service-ca\") pod \"fade9390-0055-4c73-aaab-e32a1b8e4fd3\" (UID: \"fade9390-0055-4c73-aaab-e32a1b8e4fd3\") "
Jul 24 04:10:48.283229 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:48.283207    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fade9390-0055-4c73-aaab-e32a1b8e4fd3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fade9390-0055-4c73-aaab-e32a1b8e4fd3" (UID: "fade9390-0055-4c73-aaab-e32a1b8e4fd3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:48.384428 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:48.384386    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fade9390-0055-4c73-aaab-e32a1b8e4fd3-maas-api-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:49.014284 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:49.014238    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"
Jul 24 04:10:49.014467 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:49.014291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q" event={"ID":"fade9390-0055-4c73-aaab-e32a1b8e4fd3","Type":"ContainerDied","Data":"8a2af43f49133f31cdc9e2f5f20c64d56b4ad20ab75ee2efe6ca6f454aafad75"}
Jul 24 04:10:49.014467 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:49.014336    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a2af43f49133f31cdc9e2f5f20c64d56b4ad20ab75ee2efe6ca6f454aafad75"
Jul 24 04:10:56.261221 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.261179    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"]
Jul 24 04:10:56.261688 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.261491    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn" podUID="eaa5f0d2-34c9-4c89-be95-b3c81768fe2b" containerName="maas-api" containerID="cri-o://87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947" gracePeriod=30
Jul 24 04:10:56.511883 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.511820    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:10:56.661097 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.661060    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-maas-api-tls\") pod \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") "
Jul 24 04:10:56.661337 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.661129    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-openshift-service-ca\") pod \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") "
Jul 24 04:10:56.661337 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.661178    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk8bp\" (UniqueName: \"kubernetes.io/projected/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-kube-api-access-gk8bp\") pod \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\" (UID: \"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b\") "
Jul 24 04:10:56.661557 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.661529    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eaa5f0d2-34c9-4c89-be95-b3c81768fe2b" (UID: "eaa5f0d2-34c9-4c89-be95-b3c81768fe2b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:56.663467 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.663442    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eaa5f0d2-34c9-4c89-be95-b3c81768fe2b" (UID: "eaa5f0d2-34c9-4c89-be95-b3c81768fe2b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 04:10:56.663672 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.663650    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-kube-api-access-gk8bp" (OuterVolumeSpecName: "kube-api-access-gk8bp") pod "eaa5f0d2-34c9-4c89-be95-b3c81768fe2b" (UID: "eaa5f0d2-34c9-4c89-be95-b3c81768fe2b"). InnerVolumeSpecName "kube-api-access-gk8bp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:10:56.762731 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.762638    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-openshift-service-ca\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:56.762731 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.762668    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk8bp\" (UniqueName: \"kubernetes.io/projected/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-kube-api-access-gk8bp\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:56.762731 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:56.762678    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b-maas-api-tls\") on node \"ip-10-0-132-49.ec2.internal\" DevicePath \"\""
Jul 24 04:10:57.059151 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.059059    2577 generic.go:356] "Generic (PLEG): container finished" podID="eaa5f0d2-34c9-4c89-be95-b3c81768fe2b" containerID="87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947" exitCode=0
Jul 24 04:10:57.059151 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.059111    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"
Jul 24 04:10:57.059151 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.059119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn" event={"ID":"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b","Type":"ContainerDied","Data":"87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947"}
Jul 24 04:10:57.059451 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.059168    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn" event={"ID":"eaa5f0d2-34c9-4c89-be95-b3c81768fe2b","Type":"ContainerDied","Data":"4d05f8f94ebf6c47a1583ab22f0d3a8708faf1928686800ba301469258dae1a4"}
Jul 24 04:10:57.059451 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.059192    2577 scope.go:117] "RemoveContainer" containerID="87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947"
Jul 24 04:10:57.069509 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.069487    2577 scope.go:117] "RemoveContainer" containerID="87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947"
Jul 24 04:10:57.069802 ip-10-0-132-49 kubenswrapper[2577]: E0724 04:10:57.069782    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947\": container with ID starting with 87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947 not found: ID does not exist" containerID="87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947"
Jul 24 04:10:57.069871 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.069814    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947"} err="failed to get container status \"87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947\": rpc error: code = NotFound desc = could not find container \"87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947\": container with ID starting with 87427bc68b421400ddd2d1bdab09ac25da27d97c7daadd761a3399c0278d0947 not found: ID does not exist"
Jul 24 04:10:57.082593 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.082551    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"]
Jul 24 04:10:57.085885 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.085860    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f701e0-7fc44bd94-x8jdn"]
Jul 24 04:10:57.643521 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:57.643488    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaa5f0d2-34c9-4c89-be95-b3c81768fe2b" path="/var/lib/kubelet/pods/eaa5f0d2-34c9-4c89-be95-b3c81768fe2b/volumes"
Jul 24 04:10:59.692240 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:10:59.692204    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pv8k4"]
Jul 24 04:11:00.952127 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:00.952077    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"]
Jul 24 04:11:00.954738 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:00.954709    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f701e0-5sl9q"]
Jul 24 04:11:01.642626 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:01.642596    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fade9390-0055-4c73-aaab-e32a1b8e4fd3" path="/var/lib/kubelet/pods/fade9390-0055-4c73-aaab-e32a1b8e4fd3/volumes"
Jul 24 04:11:08.542081 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:08.541972    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c598c8c48-9xnfn_3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549/authorino/0.log"
Jul 24 04:11:15.108817 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:15.108770    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f4587fcc-tmkbp_5065f5e1-78e0-4660-ade5-65c9788bd5c2/copy-manifests/0.log"
Jul 24 04:11:15.116632 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:15.116599    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f4587fcc-tmkbp_5065f5e1-78e0-4660-ade5-65c9788bd5c2/manager/0.log"
Jul 24 04:11:16.703004 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:16.702973    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8_690f73ef-e7a6-4e4f-876e-ba9c124b4f5a/util/0.log"
Jul 24 04:11:16.709875 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:16.709848    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8_690f73ef-e7a6-4e4f-876e-ba9c124b4f5a/pull/0.log"
Jul 24 04:11:16.715974 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:16.715949    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8_690f73ef-e7a6-4e4f-876e-ba9c124b4f5a/extract/0.log"
Jul 24 04:11:16.823813 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:16.823785    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29_0bd705fb-0c13-4616-ba19-d3e6b9776854/util/0.log"
Jul 24 04:11:16.831066 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:16.831040    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29_0bd705fb-0c13-4616-ba19-d3e6b9776854/pull/0.log"
Jul 24 04:11:16.837537 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:16.837504    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29_0bd705fb-0c13-4616-ba19-d3e6b9776854/extract/0.log"
Jul 24 04:11:16.954432 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:16.954343    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw_a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b/pull/0.log"
Jul 24 04:11:16.961096 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:16.961071    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw_a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b/extract/0.log"
Jul 24 04:11:16.967606 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:16.967585    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw_a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b/util/0.log"
Jul 24 04:11:17.084139 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:17.084109    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls_841c14d6-e79e-46a9-b8fc-c03df3568324/pull/0.log"
Jul 24 04:11:17.090648 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:17.090625    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls_841c14d6-e79e-46a9-b8fc-c03df3568324/extract/0.log"
Jul 24 04:11:17.097366 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:17.097337    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls_841c14d6-e79e-46a9-b8fc-c03df3568324/util/0.log"
Jul 24 04:11:17.230010 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:17.229916    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c598c8c48-9xnfn_3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549/authorino/0.log"
Jul 24 04:11:17.976195 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:17.976161    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pv8k4_5b8302a0-c23a-4e3c-af84-8f77b50dbcb6/limitador/0.log"
Jul 24 04:11:18.733663 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:18.733628    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f459d4668-2qzl2_c4d41a76-0f08-4a04-bbbe-a96749e38076/kube-auth-proxy/0.log"
Jul 24 04:11:25.687592 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.687544    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tsxt7/must-gather-8v5ls"]
Jul 24 04:11:25.688086 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.687988    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaa5f0d2-34c9-4c89-be95-b3c81768fe2b" containerName="maas-api"
Jul 24 04:11:25.688086 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.688008    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaa5f0d2-34c9-4c89-be95-b3c81768fe2b" containerName="maas-api"
Jul 24 04:11:25.688086 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.688030    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fade9390-0055-4c73-aaab-e32a1b8e4fd3" containerName="revoke-keys"
Jul 24 04:11:25.688086 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.688039    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fade9390-0055-4c73-aaab-e32a1b8e4fd3" containerName="revoke-keys"
Jul 24 04:11:25.688358 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.688137    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaa5f0d2-34c9-4c89-be95-b3c81768fe2b" containerName="maas-api"
Jul 24 04:11:25.688358 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.688151    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fade9390-0055-4c73-aaab-e32a1b8e4fd3" containerName="revoke-keys"
Jul 24 04:11:25.691297 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.691249    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsxt7/must-gather-8v5ls"
Jul 24 04:11:25.693832 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.693812    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tsxt7\"/\"default-dockercfg-jw6nv\""
Jul 24 04:11:25.693925 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.693811    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tsxt7\"/\"openshift-service-ca.crt\""
Jul 24 04:11:25.693925 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.693849    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tsxt7\"/\"kube-root-ca.crt\""
Jul 24 04:11:25.701624 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.701592    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsxt7/must-gather-8v5ls"]
Jul 24 04:11:25.797698 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.797638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/55140a82-8d90-4194-beb2-23d42b0fcc75-must-gather-output\") pod \"must-gather-8v5ls\" (UID: \"55140a82-8d90-4194-beb2-23d42b0fcc75\") " pod="openshift-must-gather-tsxt7/must-gather-8v5ls"
Jul 24 04:11:25.797698 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.797697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xlph\" (UniqueName: \"kubernetes.io/projected/55140a82-8d90-4194-beb2-23d42b0fcc75-kube-api-access-9xlph\") pod \"must-gather-8v5ls\" (UID: \"55140a82-8d90-4194-beb2-23d42b0fcc75\") " pod="openshift-must-gather-tsxt7/must-gather-8v5ls"
Jul 24 04:11:25.898708 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.898663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/55140a82-8d90-4194-beb2-23d42b0fcc75-must-gather-output\") pod \"must-gather-8v5ls\" (UID: \"55140a82-8d90-4194-beb2-23d42b0fcc75\") " pod="openshift-must-gather-tsxt7/must-gather-8v5ls"
Jul 24 04:11:25.898708 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.898707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xlph\" (UniqueName: \"kubernetes.io/projected/55140a82-8d90-4194-beb2-23d42b0fcc75-kube-api-access-9xlph\") pod \"must-gather-8v5ls\" (UID: \"55140a82-8d90-4194-beb2-23d42b0fcc75\") " pod="openshift-must-gather-tsxt7/must-gather-8v5ls"
Jul 24 04:11:25.899041 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.899021    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/55140a82-8d90-4194-beb2-23d42b0fcc75-must-gather-output\") pod \"must-gather-8v5ls\" (UID: \"55140a82-8d90-4194-beb2-23d42b0fcc75\") " pod="openshift-must-gather-tsxt7/must-gather-8v5ls"
Jul 24 04:11:25.907525 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:25.907493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xlph\" (UniqueName: \"kubernetes.io/projected/55140a82-8d90-4194-beb2-23d42b0fcc75-kube-api-access-9xlph\") pod \"must-gather-8v5ls\" (UID: \"55140a82-8d90-4194-beb2-23d42b0fcc75\") " pod="openshift-must-gather-tsxt7/must-gather-8v5ls"
Jul 24 04:11:26.000904 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:26.000796    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsxt7/must-gather-8v5ls"
Jul 24 04:11:26.138510 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:26.138484    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsxt7/must-gather-8v5ls"]
Jul 24 04:11:26.140640 ip-10-0-132-49 kubenswrapper[2577]: W0724 04:11:26.140607    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55140a82_8d90_4194_beb2_23d42b0fcc75.slice/crio-261788b6ae868fbd9d927795d1959d8066ee55ed4507a3d5dc6199aaba5fa25c WatchSource:0}: Error finding container 261788b6ae868fbd9d927795d1959d8066ee55ed4507a3d5dc6199aaba5fa25c: Status 404 returned error can't find the container with id 261788b6ae868fbd9d927795d1959d8066ee55ed4507a3d5dc6199aaba5fa25c
Jul 24 04:11:26.207458 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:26.207422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsxt7/must-gather-8v5ls" event={"ID":"55140a82-8d90-4194-beb2-23d42b0fcc75","Type":"ContainerStarted","Data":"261788b6ae868fbd9d927795d1959d8066ee55ed4507a3d5dc6199aaba5fa25c"}
Jul 24 04:11:27.217186 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:27.217147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsxt7/must-gather-8v5ls" event={"ID":"55140a82-8d90-4194-beb2-23d42b0fcc75","Type":"ContainerStarted","Data":"b3a4454da2f875517c59c5957f7a2c97a64fbb3f0e258747d593318bd555bae4"}
Jul 24 04:11:27.217714 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:27.217197    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsxt7/must-gather-8v5ls" event={"ID":"55140a82-8d90-4194-beb2-23d42b0fcc75","Type":"ContainerStarted","Data":"e4e71d5e10a1bc7f66838d82e00ebd876bed2818ad46a3afe5829829c7952ee3"}
Jul 24 04:11:27.233865 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:27.233807    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tsxt7/must-gather-8v5ls" podStartSLOduration=1.47077286 podStartE2EDuration="2.233789071s" totalImagesPullingTime="763.016211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:11:27.231994085 +0000 UTC m=+3253.992902567" watchObservedRunningTime="2026-07-24 04:11:27.233789071 +0000 UTC m=+3253.994697552"
Jul 24 04:11:28.472916 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:28.472883    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4mflr_69bd88a4-0811-4ba8-a935-20746395f8a6/global-pull-secret-syncer/0.log"
Jul 24 04:11:28.669933 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:28.669896    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wp6hh_f7984107-7911-40e1-952d-fe9561244ed4/konnectivity-agent/0.log"
Jul 24 04:11:28.692446 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:28.692410    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-49.ec2.internal_3a4fbcf9cbb15b17f76420beb7ed4f29/haproxy/0.log"
Jul 24 04:11:33.296498 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.296464    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8_690f73ef-e7a6-4e4f-876e-ba9c124b4f5a/extract/0.log"
Jul 24 04:11:33.322095 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.322044    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8_690f73ef-e7a6-4e4f-876e-ba9c124b4f5a/util/0.log"
Jul 24 04:11:33.347663 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.347608    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597ffj8_690f73ef-e7a6-4e4f-876e-ba9c124b4f5a/pull/0.log"
Jul 24 04:11:33.386057 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.386000    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29_0bd705fb-0c13-4616-ba19-d3e6b9776854/extract/0.log"
Jul 24 04:11:33.413424 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.413387    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29_0bd705fb-0c13-4616-ba19-d3e6b9776854/util/0.log"
Jul 24 04:11:33.440230 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.440196    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ldw29_0bd705fb-0c13-4616-ba19-d3e6b9776854/pull/0.log"
Jul 24 04:11:33.476073 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.476035    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw_a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b/extract/0.log"
Jul 24 04:11:33.506447 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.506410    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw_a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b/util/0.log"
Jul 24 04:11:33.534109 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.534075    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t7mtw_a3dd54b4-9deb-4ad6-bd3c-ab77f710ab9b/pull/0.log"
Jul 24 04:11:33.574571 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.574479    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls_841c14d6-e79e-46a9-b8fc-c03df3568324/extract/0.log"
Jul 24 04:11:33.599289 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.599234    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls_841c14d6-e79e-46a9-b8fc-c03df3568324/util/0.log"
Jul 24 04:11:33.630433 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:33.630387    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w5gls_841c14d6-e79e-46a9-b8fc-c03df3568324/pull/0.log"
Jul 24 04:11:34.233351 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:34.232485    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c598c8c48-9xnfn_3313e4d4-e8e0-43a1-9ce4-4c76e3c3b549/authorino/0.log"
Jul 24 04:11:34.566943 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:34.566849    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pv8k4_5b8302a0-c23a-4e3c-af84-8f77b50dbcb6/limitador/0.log"
Jul 24 04:11:36.244435 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.244387    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a9181725-f359-4f2f-970e-08ac5be9379e/alertmanager/0.log"
Jul 24 04:11:36.281290 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.280258    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a9181725-f359-4f2f-970e-08ac5be9379e/config-reloader/0.log"
Jul 24 04:11:36.304688 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.304657    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a9181725-f359-4f2f-970e-08ac5be9379e/kube-rbac-proxy-web/0.log"
Jul 24 04:11:36.330378 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.330345    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a9181725-f359-4f2f-970e-08ac5be9379e/kube-rbac-proxy/0.log"
Jul 24 04:11:36.360307 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.360259    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a9181725-f359-4f2f-970e-08ac5be9379e/kube-rbac-proxy-metric/0.log"
Jul 24 04:11:36.387145 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.387104    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a9181725-f359-4f2f-970e-08ac5be9379e/prom-label-proxy/0.log"
Jul 24 04:11:36.413188 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.413152    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a9181725-f359-4f2f-970e-08ac5be9379e/init-config-reloader/0.log"
Jul 24 04:11:36.811866 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.811820    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nnww4_0220b377-f8a0-4a99-84b4-6a9f51bdf918/node-exporter/0.log"
Jul 24 04:11:36.834146 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.834108    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nnww4_0220b377-f8a0-4a99-84b4-6a9f51bdf918/kube-rbac-proxy/0.log"
Jul 24 04:11:36.860022 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:36.859978    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nnww4_0220b377-f8a0-4a99-84b4-6a9f51bdf918/init-textfile/0.log"
Jul 24 04:11:37.331364 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:37.331335    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-748dc4bffc-n67rs_aa9ed326-99d8-44fe-93b5-9ce4f7feeae7/thanos-query/0.log"
Jul 24 04:11:37.353912 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:37.353830    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-748dc4bffc-n67rs_aa9ed326-99d8-44fe-93b5-9ce4f7feeae7/kube-rbac-proxy-web/0.log"
Jul 24 04:11:37.376719 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:37.376687    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-748dc4bffc-n67rs_aa9ed326-99d8-44fe-93b5-9ce4f7feeae7/kube-rbac-proxy/0.log"
Jul 24 04:11:37.398353 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:37.398320    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-748dc4bffc-n67rs_aa9ed326-99d8-44fe-93b5-9ce4f7feeae7/prom-label-proxy/0.log"
Jul 24 04:11:37.420842 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:37.420809    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-748dc4bffc-n67rs_aa9ed326-99d8-44fe-93b5-9ce4f7feeae7/kube-rbac-proxy-rules/0.log"
Jul 24 04:11:37.444626 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:37.444578    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-748dc4bffc-n67rs_aa9ed326-99d8-44fe-93b5-9ce4f7feeae7/kube-rbac-proxy-metrics/0.log"
Jul 24 04:11:38.608420 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.608389    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-sxk8t_efaf2452-d67f-4599-998a-4a496d2c9c07/networking-console-plugin/0.log"
Jul 24 04:11:38.662928 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.662876    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"]
Jul 24 04:11:38.670853 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.670823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.678007 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.677979    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"]
Jul 24 04:11:38.734789 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.734754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p245r\" (UniqueName: \"kubernetes.io/projected/590922f4-12e9-460c-aafe-f4be25dcf242-kube-api-access-p245r\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.735085 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.735060    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-proc\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.735360 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.735341    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-sys\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.735550 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.735534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-lib-modules\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.735693 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.735680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-podres\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.836557 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.836520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-sys\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.836738 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.836594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-lib-modules\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.836738 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.836623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-podres\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.836738 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.836647    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-sys\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.836738 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.836667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p245r\" (UniqueName: \"kubernetes.io/projected/590922f4-12e9-460c-aafe-f4be25dcf242-kube-api-access-p245r\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.836941 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.836760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-proc\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.836941 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.836808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-lib-modules\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.836941 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.836808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-podres\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.836941 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.836868    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/590922f4-12e9-460c-aafe-f4be25dcf242-proc\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.845044 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.845015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p245r\" (UniqueName: \"kubernetes.io/projected/590922f4-12e9-460c-aafe-f4be25dcf242-kube-api-access-p245r\") pod \"perf-node-gather-daemonset-c7fcr\" (UID: \"590922f4-12e9-460c-aafe-f4be25dcf242\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:38.989991 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:38.989938    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:39.176427 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:39.175595    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"]
Jul 24 04:11:39.323644 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:39.323598    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr" event={"ID":"590922f4-12e9-460c-aafe-f4be25dcf242","Type":"ContainerStarted","Data":"3389f680d6bd0aa64a1d5c26be30c394a3aa80587b807ea270f068041e4bebe3"}
Jul 24 04:11:39.323820 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:39.323673    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr" event={"ID":"590922f4-12e9-460c-aafe-f4be25dcf242","Type":"ContainerStarted","Data":"13c52deb4c4b07b1592b096b4d339df4d4c30fe79be5f45d8c1d5804813e1828"}
Jul 24 04:11:39.324259 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:39.324223    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:39.342852 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:39.342783    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr" podStartSLOduration=1.342762062 podStartE2EDuration="1.342762062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 04:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:11:39.339109888 +0000 UTC m=+3266.100018369" watchObservedRunningTime="2026-07-24 04:11:39.342762062 +0000 UTC m=+3266.103670542"
Jul 24 04:11:39.796961 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:39.796928    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6447457645-26pxx_ad74ecb7-3c84-4ee1-965d-10686ed5eddd/console/0.log"
Jul 24 04:11:41.490910 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:41.490879    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xp58_6cd4908e-c044-40f4-88b4-6606c01d3054/dns/0.log"
Jul 24 04:11:41.528171 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:41.528132    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xp58_6cd4908e-c044-40f4-88b4-6606c01d3054/kube-rbac-proxy/0.log"
Jul 24 04:11:41.606998 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:41.606962    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8b2k6_49dc3ed5-5da8-4347-b531-d650944e4ee8/dns-node-resolver/0.log"
Jul 24 04:11:42.320462 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:42.320430    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lxx2x_d03b8479-3f64-4961-8243-96e9d6f6d551/node-ca/0.log"
Jul 24 04:11:43.426716 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:43.426671    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f459d4668-2qzl2_c4d41a76-0f08-4a04-bbbe-a96749e38076/kube-auth-proxy/0.log"
Jul 24 04:11:44.506191 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:44.506152    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prsrs_8b304cd9-4445-4be1-921f-0797a250e25d/serve-healthcheck-canary/0.log"
Jul 24 04:11:45.002460 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:45.002408    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-km4rg_cc7c47fc-3227-4aec-bf89-07c4da81121c/kube-rbac-proxy/0.log"
Jul 24 04:11:45.032581 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:45.032548    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-km4rg_cc7c47fc-3227-4aec-bf89-07c4da81121c/exporter/0.log"
Jul 24 04:11:45.062471 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:45.062425    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-km4rg_cc7c47fc-3227-4aec-bf89-07c4da81121c/extractor/0.log"
Jul 24 04:11:46.348958 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:46.348923    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-c7fcr"
Jul 24 04:11:48.039626 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:48.039595    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f4587fcc-tmkbp_5065f5e1-78e0-4660-ade5-65c9788bd5c2/manager/0.log"
Jul 24 04:11:48.063856 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:48.063817    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f4587fcc-tmkbp_5065f5e1-78e0-4660-ade5-65c9788bd5c2/copy-manifests/0.log"
Jul 24 04:11:49.410735 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:49.410699    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-54698f7c78-rrv9j_008a9dec-e4ec-46e3-9f74-1d60d68b5cee/manager/0.log"
Jul 24 04:11:49.711026 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:49.710932    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f459d4668-2qzl2_c4d41a76-0f08-4a04-bbbe-a96749e38076/kube-auth-proxy/0.log"
Jul 24 04:11:51.430405 ip-10-0-132-49 kubenswrapper[2577]: I0724 04:11:51.430374    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f459d4668-2qzl2_c4d41a76-0f08-4a04-bbbe-a96749e38076/kube-auth-proxy/0.log"